Oct 30 17:53:51.485816 ip-10-0-131-31 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 17:53:51.485831 ip-10-0-131-31 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 17:53:51.485841 ip-10-0-131-31 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 17:53:51.486177 ip-10-0-131-31 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 17:54:01.734909 ip-10-0-131-31 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 17:54:01.734927 ip-10-0-131-31 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 86d2350389e6498cb24d7d9562e4c6f3 --
Oct 30 17:55:56.343160 ip-10-0-131-31 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 17:55:56.742370 ip-10-0-131-31 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 17:55:56.742370 ip-10-0-131-31 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 17:55:56.742370 ip-10-0-131-31 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 17:55:56.742370 ip-10-0-131-31 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 17:55:56.747093 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.746998    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:55:56.747093 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747088    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:55:56.747093 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747094    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:55:56.747093 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747098    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:55:56.747093 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747102    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747109    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747113    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747118    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747122    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747126    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747130    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747134    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747138    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747142    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747147    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747152    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747156    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747160    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747165    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747169    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747174    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747182    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747187    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747191    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:55:56.747396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747196    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747201    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747205    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747209    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747215    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747219    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747223    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747228    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747232    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747238    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747243    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747247    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747252    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747257    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747262    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747267    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747272    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747277    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747281    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:55:56.748253 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747286    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747292    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747296    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747302    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747307    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747311    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747316    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747321    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747327    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747332    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747341    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747349    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747354    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747359    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747364    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747368    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747373    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747378    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747383    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:55:56.749077 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747388    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747392    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747396    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747401    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747405    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747409    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747413    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747419    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747424    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747429    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747434    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747439    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747443    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747449    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747453    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747458    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747462    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747467    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747472    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747476    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:55:56.749945 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747481    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747487    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747492    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747497    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747502    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747507    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747511    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747516    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747520    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747525    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747530    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747535    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747540    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747545    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747549    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747554    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747559    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747563    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747568    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:55:56.750793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747573    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747579    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747583    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747588    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.747595    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.748957    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.748970    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.748975    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.748981    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.748986    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.748991    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.748995    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749000    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749005    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749010    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749016    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749022    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749026    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749031    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749035    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:55:56.751392 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749040    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749045    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749052    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749060    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749065    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749070    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749074    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749079    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749083    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749087    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749092    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749097    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749101    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749107    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749112    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749117    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749122    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749126    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749131    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:55:56.752150 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749136    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749140    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749145    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749149    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749154    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749159    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749164    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749168    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749173    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749179    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749184    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749189    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749193    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749199    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749203    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749209    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749214    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749218    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749223    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749228    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:55:56.753008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749233    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749238    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749244    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749249    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749253    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749258    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749263    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749267    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749272    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749277    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749282    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749286    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749291    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749296    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749300    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749305    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749309    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749314    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749318    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:55:56.753865 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749323    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749328    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749334    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749340    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749345    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749350    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749354    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749359    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749364    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749370    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749375    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749380    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749384    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749389    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749394    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749399    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749404    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749409    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749414    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:55:56.754464 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749418    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749423    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749428    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749432    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749437    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749442    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749446    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749451    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749455    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749460    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749467    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749476    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749482    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.749487    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750383    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750394    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750401    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750406    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750411    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750416    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:55:56.755194 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750421    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750427    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750432    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750437    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750442    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750446    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750451    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750457    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750462    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750466    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750470    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750475    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750480    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750485    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750490    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750494    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750499    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750504    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750508    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750513    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:55:56.755741 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750518    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750522    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750527    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750531    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750536    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750541    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750545    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750550    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750554    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750560    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750565    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750569    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750574    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750581    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750585    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750590    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750595    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750600    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750605    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:55:56.756266 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750610    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750615    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750619    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750624    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750628    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750633    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750638    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750643    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750647    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750652    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750657    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750661    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750666    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750671    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750676    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750681    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750685    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750690    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750695    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750699    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:55:56.756793 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750704    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750708    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750713    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750718    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750723    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750728    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750734    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750738    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750743    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750750    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750756    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750763    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750768    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750773    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750778    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750782    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750787    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750791    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750795    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:55:56.757307 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750800    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750804    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750809    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750813    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750818    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750823    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750828    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750832    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750837    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750842    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750855    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750860    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750865    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750869    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750874    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750878    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750883    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750891    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750897    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:55:56.757752 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750922    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750929    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.750934    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751573    2576 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751588    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751601    2576 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751609    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751623    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751629    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751637    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751644    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751649    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751654    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751660    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751666    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751672    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751677    2576 flags.go:64] FLAG: --cgroup-root=""
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751682    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751688    2576 flags.go:64] FLAG: --client-ca-file=""
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751692    2576 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751697    2576 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751704    2576 flags.go:64] FLAG: --cluster-domain=""
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751710    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 17:55:56.758214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751715    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751721    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751727    2576 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751734    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751741    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751747    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751752    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751757    2576 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751764    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751771    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751776    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751782    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751789    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751794    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751800    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751805    2576 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751810    2576 flags.go:64] FLAG: --enable-server="true"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751815    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751824    2576 flags.go:64] FLAG: --event-burst="100"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751830    2576 flags.go:64] FLAG: --event-qps="50"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751835    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751840    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751845    2576 flags.go:64] FLAG: --eviction-hard=""
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751853    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 17:55:56.758748 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751858    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751864    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751869    2576 flags.go:64] FLAG: --eviction-soft=""
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751874    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751879    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751884    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751889    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751894    2576 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751899    2576 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751927    2576 flags.go:64] FLAG: --feature-gates=""
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751935    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751940    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751948    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751954    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751959    2576 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751964    2576 flags.go:64] FLAG: --help="false"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751969    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-131-31.ec2.internal"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751979    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751984    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751989    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.751995    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752001    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752007    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 17:55:56.759427 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752012    2576 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752017    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752022    2576 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752027    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752032    2576 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752037    2576 flags.go:64] FLAG: --kube-reserved=""
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752043    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752048    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752054    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752058    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752063    2576 flags.go:64] FLAG: --lock-file=""
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752068    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752073    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752078    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752088    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752093    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752098    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752103    2576 flags.go:64] FLAG: --logging-format="text"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752108    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752114    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752120    2576 flags.go:64] FLAG: --manifest-url=""
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752128    2576 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752136    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752142    2576 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752148    2576 flags.go:64] FLAG: --max-pods="110"
Oct 30 17:55:56.760225 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752153    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752158    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752165    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752170    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752175    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752180    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752185    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752198    2576 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752203    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752208    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752213    2576 flags.go:64] FLAG: --pod-cidr=""
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752218    2576 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752223    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752228    2576 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752234    2576 flags.go:64] FLAG: --port="10250"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752239    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752244    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02f3a9b1604f76372"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752250    2576 flags.go:64] FLAG: --qos-reserved=""
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752255    2576 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752260    2576 flags.go:64] FLAG: --register-node="true"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752265    2576 flags.go:64] FLAG: --register-with-taints=""
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752272    2576 flags.go:64] FLAG: --registry-burst="10"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752277    2576 flags.go:64] FLAG: --registry-qps="5"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752282    2576 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752287    2576 flags.go:64] FLAG: --reserved-memory=""
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752294    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 17:55:56.761420 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752299    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752304    2576 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752309    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752315    2576 flags.go:64] FLAG: --runonce="false"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752320    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752325    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752331    2576 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752336    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752341    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752347    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752352    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752358    2576 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752363    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752368    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752373    2576 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752378    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752383    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752388    2576 flags.go:64] FLAG: --system-cgroups=""
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752393    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752403    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752408    2576 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752413    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752420    2576 flags.go:64] FLAG: --tls-min-version=""
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752425    2576 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752429    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 17:55:56.762562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752434    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752439    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752444    2576 flags.go:64] FLAG: --v="2"
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752452    2576 flags.go:64] FLAG: --version="false"
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752459    2576 flags.go:64] FLAG: --vmodule=""
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752465    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.752471    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752693    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752701    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752706    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752711    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752716    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752721    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752726    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752731    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752736    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752741    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752747    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752754    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752760    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752766    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:55:56.763688 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752770    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752775    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752780    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752784    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752789    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752794    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752798    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752803    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752808    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752813    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752818    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752823    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752827    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752832    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752837    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752842    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752846    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752851    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752855    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752860    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:55:56.764543 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752865    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752870    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752875    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752879    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752884    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752889    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752893    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752898    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752922    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752928    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752933    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752938    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752943    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752948    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752952    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752957    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752963    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752970    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752975    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:55:56.765192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752980    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752985    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752990    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752995    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.752999    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753004    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753008    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753015    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753020    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753024    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753028    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753033    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753037    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753042    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753047    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753052    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753057    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753061    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753066    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:55:56.766005 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753071    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753075    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753080    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753085    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753090    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753096    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753101    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753105    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753110    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753115    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753119    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753123    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753128    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753132    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753137    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753142    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753147    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753151    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753156    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753160    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:55:56.766635 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753167    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753172    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753176    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753181    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753187    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753194    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753199    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753203    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753207    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753212    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753217    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753222    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753227    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753231    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.753276    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753337    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:55:56.767285 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753344    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753351    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753355    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753360    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753364    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753369    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753374    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753379    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753384    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753388    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753393    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753398    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753402    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753407    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753411    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753416    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753423    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753428    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753432    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753437    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:55:56.767738 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753442    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753448    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753457    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753463    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753468    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753472    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753477    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753482    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753486    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753491    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753495    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753501    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753505    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753510    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753515    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753520    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753525    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753529    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753534    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:55:56.768345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753538    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753543    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753548    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753552    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753557    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753562    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753566    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753571    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753575    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753583    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753588    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753593    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753598    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753602    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753607    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753614    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753618    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753623    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753628    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753632    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:55:56.768792 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753637    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753642    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753646    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753651    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753656    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753661    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753665    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753670    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753675    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753681    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753686    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753691    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753695    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753700    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753704    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753709    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753714    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753718    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753723    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753728    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:55:56.769347 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753732    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753738    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753743    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753747    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753752    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753757    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753761    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753767    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753772    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753780    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753786    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753791    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753795    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753800    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753805    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753810    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753814    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753819    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753824    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:55:56.769807 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753828    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753833    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753841    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753846    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753850    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753855    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.753860    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.762154    2576 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.762171    2576 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.762190    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.762198    2576 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762282    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762290    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762295    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762301    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762307    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762312    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762317    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762322    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:55:56.770536 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762327    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762332    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762337    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762341    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762346    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762350    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762356    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762360    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762365    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762370    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762374    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762379    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762384    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762388    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762393    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762398    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762403    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762408    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762413    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762417    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:55:56.771345 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762422    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762426    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762431    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762436    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762440    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762445    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762450    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762455    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762459    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762464    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762469    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762473    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762478    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762483    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762487    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762492    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762497    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762502    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762507    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:55:56.772081 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762512    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762517    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762521    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762525    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762530    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762535    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762540    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762544    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762549    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762555    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762559    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762564    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762568    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762573    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762577    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762582    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762588    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762595    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762600    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:55:56.772637 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762606    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762611    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762617    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762622    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762627    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762631    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762636    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762641    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762646    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762651    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762655    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762660    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762665    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762670    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762674    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762679    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762684    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762689    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762694    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762698    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:55:56.773396 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762703    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762708    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762713    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762718    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762723    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762727    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762732    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762737    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762742    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762746    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762751    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762756    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762760    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762765    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762770    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762774    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762781    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762788    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762793    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:55:56.773881 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.762798    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.762839    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763092    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763100    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763106    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763111    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763116    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763120    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763125    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763130    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763135    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763140    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763145    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763150    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763154    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:55:56.774370 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763159    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763164    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763170    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763177    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763183    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763188    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763193    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763198    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763203    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763208    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763212    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763217    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763222    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763227    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763232    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763236    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763241    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763246    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763251    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:55:56.774893 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763255    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763260    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763264    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763269    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763274    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763279    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763283    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763288    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763292    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763297    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763302    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763306    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763311    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763315    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763320    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763325    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763331    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763336    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763340    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763344    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:55:56.775695 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763349    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763354    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763359    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763363    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763371    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763376    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763381    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763385    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763390    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763394    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763399    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763404    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763409    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763413    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763418    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763423    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763427    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763432    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763437    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763441    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:55:56.776261 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763446    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763451    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763455    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763461    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763465    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763470    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763474    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763479    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763484    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763490    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763494    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763499    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763504    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763508    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763513    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763517    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763524    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763529    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763534    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:55:56.776746 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763538    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763543    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763548    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763552    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763557    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763562    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763566    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763571    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763576    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763581    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763588    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763595    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763601    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763606    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763611    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:55:56.777282 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.763652    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763704    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763710    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763716    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763721    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763725    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763730    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763736    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763742    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763747    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763751    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763756    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763761    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763765    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763771    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763776    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763781    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763785    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763790    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763795    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:55:56.777682 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763800    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763804    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763809    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763813    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763818    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763822    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763827    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763832    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763836    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763841    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763846    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763851    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763855    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763860    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763865    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763869    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763874    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763879    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763884    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763890    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:55:56.778254 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763895    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763900    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763923    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763928    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763933    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763937    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763944    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763949    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763953    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763959    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763963    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763968    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763973    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763978    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763982    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763987    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763991    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.763996    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764001    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764006    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:55:56.778713 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764011    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764016    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764020    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764025    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764029    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764033    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764038    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764043    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764048    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764052    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764057    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764063    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764068    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764072    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764078    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764083    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764088    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764093    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764099    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764105    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:55:56.779237 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764109    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764115    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764120    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764127    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764135    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764141    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764147    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764152    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764157    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764161    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764166    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764171    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764176    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764180    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764185    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764190    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764194    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764199    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:55:56.779701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764204    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764209    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764214    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764218    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764223    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764229    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764238    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764244    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:56.764249    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.764327    2576 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.766973    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.768168    2576 server.go:1014] "Starting client certificate rotation"
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.768271    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.768301    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.770603    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 17:55:56.780143 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.774557    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 17:55:56.785606 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.785590    2576 log.go:25] "Validated CRI v1 runtime API"
Oct 30 17:55:56.788835 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.788814    2576 log.go:25] "Validated CRI v1 image API"
Oct 30 17:55:56.789675 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.789647    2576 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 17:55:56.796791 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.796761    2576 fs.go:135] Filesystem UUIDs: map[475bd308-c0c5-4f3a-9f15-044b048b51f7:/dev/nvme0n1p4 49d15a24-7cfa-4a97-abbc-3dc9814f3948:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 30 17:55:56.796869 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.796789    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 17:55:56.802003 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.801856    2576 manager.go:217] Machine: {Timestamp:2026-10-30 17:55:56.800530992 +0000 UTC m=+0.343142426 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3160287 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2df2e1f4aa427798545250ddd22641 SystemUUID:ec2df2e1-f4aa-4277-9854-5250ddd22641 BootID:86d23503-89e6-498c-b24d-7d9562e4c6f3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ce:36:0c:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ce:36:0c:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:71:b5:12:ea:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 17:55:56.802461 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.802448    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 17:55:56.802567 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.802551    2576 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 17:55:56.805376 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.805349    2576 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 17:55:56.805532 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.805372    2576 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-31.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 17:55:56.805955 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.805945    2576 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 17:55:56.805992 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.805956    2576 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 17:55:56.805992 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.805983    2576 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 17:55:56.806044 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.805995    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 17:55:56.806798 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.806788    2576 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 17:55:56.810511 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.810500    2576 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 17:55:56.810624 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.810615    2576 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 17:55:56.811224 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.811215    2576 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 17:55:56.811254 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.811228    2576 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 17:55:56.811254 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.811238    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 17:55:56.811254 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.811247    2576 kubelet.go:405] "Adding apiserver pod source"
Oct 30 17:55:56.811254 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.811254    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 17:55:56.811671 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.811652    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 17:55:56.814036 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.814020    2576 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 17:55:56.818802 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.818789    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 17:55:56.819679 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.819668    2576 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 17:55:56.819725 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.819698    2576 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 17:55:56.819850 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.819841    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 17:55:56.819881 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.819857    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 17:55:56.819881 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.819863    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 17:55:56.819881 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.819868    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 17:55:56.819881 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.819874    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 17:55:56.819881 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.819880    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 17:55:56.820632 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.820623    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 17:55:56.820660 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.820633    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 17:55:56.820660 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.820641    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 17:55:56.820660 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.820647    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 17:55:56.820660 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.820659    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 17:55:56.820824 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.820667    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 17:55:56.821353 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.821344    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 17:55:56.821383 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.821354    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 17:55:56.824917 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.824893    2576 server.go:1263] "Started kubelet"
Oct 30 17:55:56.825049 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.825014    2576 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 17:55:56.825136 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.825063    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 17:55:56.825136 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.825132    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 17:55:56.825598 ip-10-0-131-31 systemd[1]: Started Kubernetes Kubelet.
Oct 30 17:55:56.829248 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.829019    2576 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 17:55:56.833840 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.833796    2576 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 17:55:56.835632 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.835607    2576 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 17:55:56.836309 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.836168    2576 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 17:55:56.836309 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.836182    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 17:55:56.836747 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.836725    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:56.836835 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.836771    2576 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 17:55:56.836921 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.836895    2576 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 17:55:56.837098 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.836792    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 17:55:56.837389 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837376    2576 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 17:55:56.837463 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837454    2576 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 17:55:56.837549 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837476    2576 factory.go:55] Registering systemd factory
Oct 30 17:55:56.837649 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837556    2576 factory.go:223] Registration of the systemd container factory successfully
Oct 30 17:55:56.837755 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837742    2576 factory.go:153] Registering CRI-O factory
Oct 30 17:55:56.837820 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837759    2576 factory.go:223] Registration of the crio container factory successfully
Oct 30 17:55:56.837875 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837829    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 17:55:56.837875 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837849    2576 factory.go:103] Registering Raw factory
Oct 30 17:55:56.837875 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.837865    2576 manager.go:1199] Started watching for new ooms in manager
Oct 30 17:55:56.838936 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.838857    2576 manager.go:319] Starting recovery of all containers
Oct 30 17:55:56.847736 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.847723    2576 manager.go:324] Recovery completed
Oct 30 17:55:56.851697 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.851677    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:55:56.854842 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.854820    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:55:56.854936 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.854865    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:55:56.854936 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.854880    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:55:56.855444 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.855423    2576 cpu_manager.go:226] "Starting" policy="none"
Oct 30 17:55:56.855444 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.855442    2576 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 17:55:56.855552 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.855463    2576 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 17:55:56.855552 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.855433    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:55:56.856236 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.856221    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:55:56.856318 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.856252    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:55:56.856318 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.856266    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:55:56.857628 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.857616    2576 policy_none.go:50] "Start"
Oct 30 17:55:56.857676 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.857632    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 17:55:56.857676 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.857642    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 17:55:56.859564 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.859553    2576 policy_none.go:44] "Start"
Oct 30 17:55:56.863985 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.862899    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-31.ec2.internal.18e3604c27c9143c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-31.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-31.ec2.internal,},FirstTimestamp:2026-10-30 17:55:56.824859708 +0000 UTC m=+0.367471134,LastTimestamp:2026-10-30 17:55:56.824859708 +0000 UTC m=+0.367471134,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-31.ec2.internal,}"
Oct 30 17:55:56.866961 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.866944    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 17:55:56.867016 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.866979    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xqv87"
Oct 30 17:55:56.874339 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.874314    2576 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-31.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 17:55:56.874437 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.874353    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-31.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 17:55:56.878303 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.878285    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xqv87"
Oct 30 17:55:56.881313 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.881027    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-31.ec2.internal.18e3604c29928fa3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-31.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-31.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-31.ec2.internal,},FirstTimestamp:2026-10-30 17:55:56.854841251 +0000 UTC m=+0.397452677,LastTimestamp:2026-10-30 17:55:56.854841251 +0000 UTC m=+0.397452677,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-31.ec2.internal,}"
Oct 30 17:55:56.881423 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.881331    2576 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-31.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 17:55:56.881423 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.881147    2576 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.902227    2576 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.904810    2576 manager.go:341] "Starting Device Plugin manager"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.904874    2576 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.904886    2576 server.go:87] "Starting device plugin registration server"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.905184    2576 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.905196    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.905326    2576 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.905436    2576 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.905445    2576 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.905964    2576 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.906030    2576 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.909422    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.910830    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.912069    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.912087    2576 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.912105    2576 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:56.912142    2576 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 17:55:56.926859 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:56.925829    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:55:57.005935 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.005840    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:55:57.006810 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.006794    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:55:57.006919 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.006824    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:55:57.006919 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.006836    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:55:57.006919 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.006872    2576 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.013036 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.013020    2576 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"]
Oct 30 17:55:57.013091 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.013081    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:55:57.013773 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.013759    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:55:57.013829 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.013785    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:55:57.013829 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.013795    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:55:57.016001 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.015989    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:55:57.016147 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.016132    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.016186 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.016163    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:55:57.016738 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.016715    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:55:57.016816 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.016744    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:55:57.016816 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.016752    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:55:57.016816 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.016758    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:55:57.016816 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.016769    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:55:57.016816 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.016778    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:55:57.018872 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.018856    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.018964 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.018888    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:55:57.019598 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.019582    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:55:57.019681 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.019608    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:55:57.019681 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.019623    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:55:57.026089 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.026072    2576 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.026171 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.026092    2576 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-31.ec2.internal\": node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.038457 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.038438    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.038525 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.038469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.038525 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.038490    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167496dc01702046817db5cdac7ef75c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-31.ec2.internal\" (UID: \"167496dc01702046817db5cdac7ef75c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.047440 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.047418    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-31.ec2.internal\" not found" node="ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.050183 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.050165    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-31.ec2.internal\" not found" node="ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.066278 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.066255    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.139082 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.139062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.139082 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.139086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.139249 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.139102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167496dc01702046817db5cdac7ef75c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-31.ec2.internal\" (UID: \"167496dc01702046817db5cdac7ef75c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.139249 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.139145    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167496dc01702046817db5cdac7ef75c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-31.ec2.internal\" (UID: \"167496dc01702046817db5cdac7ef75c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.139249 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.139171    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.139249 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.139168    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.167135 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.167115    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.268034 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.267968    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.350683 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.350653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.353823 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.353795    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:57.368964 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.368943    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.469485 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.469443    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.570004 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.569924    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.670530 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.670498    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.768019 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.767986    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 17:55:57.768475 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.768147    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 17:55:57.768475 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.768176    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 17:55:57.771140 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.771107    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.872232 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.872203    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:57.876405 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.876379    2576 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:55:57.883449 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.883393    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 17:50:57 +0000 UTC" deadline="2028-04-09 02:44:57.593580728 +0000 UTC"
Oct 30 17:55:57.883449 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.883448    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12632h48m59.71013561s"
Oct 30 17:55:57.884488 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.884474    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:55:57.885718 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.885700    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:55:57.885791 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.885739    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:55:57.885791 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.885756    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:55:57.889284 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:57.889258    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167496dc01702046817db5cdac7ef75c.slice/crio-61284ced422bbbce93df64f4c4a520ec2d52427cc95ee8ec7885a6993a7ebd53 WatchSource:0}: Error finding container 61284ced422bbbce93df64f4c4a520ec2d52427cc95ee8ec7885a6993a7ebd53: Status 404 returned error can't find the container with id 61284ced422bbbce93df64f4c4a520ec2d52427cc95ee8ec7885a6993a7ebd53
Oct 30 17:55:57.898344 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.898328    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 17:55:57.898555 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:57.898538    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8158fd6db0c3186f36b1583fbfe4e93d.slice/crio-94750cd6b25ccb4311f07f812adc895276b1fa831e3df0a40f14703974234cd3 WatchSource:0}: Error finding container 94750cd6b25ccb4311f07f812adc895276b1fa831e3df0a40f14703974234cd3: Status 404 returned error can't find the container with id 94750cd6b25ccb4311f07f812adc895276b1fa831e3df0a40f14703974234cd3
Oct 30 17:55:57.915529 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.915494    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal" event={"ID":"167496dc01702046817db5cdac7ef75c","Type":"ContainerStarted","Data":"61284ced422bbbce93df64f4c4a520ec2d52427cc95ee8ec7885a6993a7ebd53"}
Oct 30 17:55:57.916504 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:57.916486    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal" event={"ID":"8158fd6db0c3186f36b1583fbfe4e93d","Type":"ContainerStarted","Data":"94750cd6b25ccb4311f07f812adc895276b1fa831e3df0a40f14703974234cd3"}
Oct 30 17:55:57.972681 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:57.972655    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:58.073187 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:58.073157    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:58.173693 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:58.173616    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:58.274320 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:58.274288    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:55:58.340289 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.340256    2576 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:55:58.438086 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.438016    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:58.463878 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.463851    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 17:55:58.465255 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.465228    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:55:58.489440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.489416    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 17:55:58.812576 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.812494    2576 apiserver.go:52] "Watching apiserver"
Oct 30 17:55:58.836622 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.836580    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 17:55:58.863749 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.863718    2576 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 17:55:58.864364 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.864333    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-vxvtw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal","openshift-network-diagnostics/network-check-target-d9ml4","openshift-network-operator/iptables-alerter-6s62t","kube-system/global-pull-secret-syncer-trnpq","kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal","openshift-multus/multus-7zqrb","openshift-multus/multus-additional-cni-plugins-lfmxr","openshift-multus/network-metrics-daemon-dttj5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk","openshift-ovn-kubernetes/ovnkube-node-krkmg","kube-system/konnectivity-agent-s6687","openshift-cluster-node-tuning-operator/tuned-hmlkj","openshift-dns/node-resolver-lxg8q"]
Oct 30 17:55:58.867221 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.867194    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.869535 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.869509    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.871944 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.871897    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:55:58.872053 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:58.872014    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:55:58.874368 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.874348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:58.877173 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.877039    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:58.877173 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:58.877111    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:55:58.879736 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.879385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:58.881842 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.881819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:55:58.881976 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:58.881891    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:55:58.884247 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.884225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:58.888094 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.888074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:58.888970 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.888947    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 17:55:58.889124 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.889094    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 17:55:58.889189 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.889156    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 17:55:58.889189 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.889159    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-8vdz2\""
Oct 30 17:55:58.889282 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.889242    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 17:55:58.889347 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.889334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:58.890049 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.890030    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pc6fd\""
Oct 30 17:55:58.892220 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.892199    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.898007 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.897982    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:55:58.902976 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.902952    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 17:55:58.903089 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.903037    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w885q"
Oct 30 17:55:58.903089 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.903068    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w885q"
Oct 30 17:55:58.938198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.938167    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 17:55:58.948027 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.948003    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:55:58.949130 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.949110    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:55:58.950799 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.950771    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-host\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.950966 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.950819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.950966 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.950841    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-iptables-alerter-script\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:58.950966 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.950859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5424\" (UniqueName: \"kubernetes.io/projected/6fe540a2-725b-4723-bb8c-11b83afe9b2a-kube-api-access-p5424\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.950966 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.950887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:58.950966 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.950924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hq7z\" (UniqueName: \"kubernetes.io/projected/04e386bc-2d2b-438e-bb10-af8b05ea5b30-kube-api-access-8hq7z\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.950967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.950991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-run\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-dbus\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-netd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-env-overrides\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951108    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04e386bc-2d2b-438e-bb10-af8b05ea5b30-hosts-file\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-binary-copy\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-system-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-netns\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-bin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.951228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-host-slash\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-tuned\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-kubelet\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-systemd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951367    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-socket-dir-parent\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-hostroot\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951438    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-systemd\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-multus\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-multus-certs\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-socket-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-config\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951597    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c3da85c-aef5-4f3f-9381-8665d944c858-ovn-node-metrics-cert\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysconfig\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951638    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.951776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc6lk\" (UniqueName: \"kubernetes.io/projected/0c3da85c-aef5-4f3f-9381-8665d944c858-kube-api-access-vc6lk\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951680    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cnibin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lvb4\" (UniqueName: \"kubernetes.io/projected/dc9f7717-0e17-4ca2-aaa2-286a398e3435-kube-api-access-4lvb4\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-conf\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951756    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951772    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-sys-fs\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-script-lib\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vqpq\" (UniqueName: \"kubernetes.io/projected/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-kube-api-access-8vqpq\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951857    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwf77\" (UniqueName: \"kubernetes.io/projected/1e3c695e-3fac-49f4-838e-251b0c17fe67-kube-api-access-vwf77\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951894    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a304069c-7b5f-4ddf-9549-33797ac13658-agent-certs\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.951962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-var-lib-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cnibin\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cni-binary-copy\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.952596 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952126    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-sys\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-slash\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-log-socket\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-bin\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-k8s-cni-cncf-io\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-kubelet-config\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-node-log\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th8x7\" (UniqueName: \"kubernetes.io/projected/91a3053e-bb6f-4678-b865-4bb0e24f7dad-kube-api-access-th8x7\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952399    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khb5q\" (UniqueName: \"kubernetes.io/projected/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-kube-api-access-khb5q\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952416    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-conf-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-lib-modules\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-device-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9j2g\" (UniqueName: \"kubernetes.io/projected/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kube-api-access-r9j2g\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a3053e-bb6f-4678-b865-4bb0e24f7dad-serviceca\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952509    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a304069c-7b5f-4ddf-9549-33797ac13658-konnectivity-ca\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:55:58.953395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-etc-kubernetes\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-var-lib-kubelet\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-netns\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-ovn\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e386bc-2d2b-438e-bb10-af8b05ea5b30-tmp-dir\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-registration-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-systemd-units\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952680    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952697    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-daemon-config\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-kubernetes\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952752    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-tmp\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-etc-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a3053e-bb6f-4678-b865-4bb0e24f7dad-host\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-kubelet\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:58.954198 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.952835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-modprobe-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:58.996489 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996460    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 17:55:58.996667 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996546    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 17:55:58.996667 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996567    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 17:55:58.996667 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996617    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 17:55:58.996836 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996678    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v7j5r\""
Oct 30 17:55:58.996836 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996776    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 17:55:58.996836 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996832    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 17:55:58.997013 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996836    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 17:55:58.997013 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996930    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 17:55:58.997013 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996971    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 17:55:58.997013 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997007    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 17:55:58.997222 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997079    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 17:55:58.997222 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997090    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 17:55:58.997222 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997120    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 17:55:58.997222 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997161    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z9xqn\""
Oct 30 17:55:58.997222 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997180    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 17:55:58.997222 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997208    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997227    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997258    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vkdz6\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997277    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996931    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997338    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997376    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997260    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.996779    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zxlb9\""
Oct 30 17:55:58.997465 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:58.997417    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 17:55:59.000675 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.000653    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mmrm7\""
Oct 30 17:55:59.000803 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.000676    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-26whl\""
Oct 30 17:55:59.012757 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.012729    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 17:55:59.012875 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.012804    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-z6vc6\""
Oct 30 17:55:59.053838 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-kubelet\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.053838 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-systemd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054042 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054042 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-socket-dir-parent\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.054042 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-hostroot\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.054042 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-systemd\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.054042 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.054042 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053958    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-multus\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.054042 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.053982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-multus-certs\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.054042 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-socket-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-config\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054072    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c3da85c-aef5-4f3f-9381-8665d944c858-ovn-node-metrics-cert\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysconfig\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc6lk\" (UniqueName: \"kubernetes.io/projected/0c3da85c-aef5-4f3f-9381-8665d944c858-kube-api-access-vc6lk\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cnibin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lvb4\" (UniqueName: \"kubernetes.io/projected/dc9f7717-0e17-4ca2-aaa2-286a398e3435-kube-api-access-4lvb4\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-conf\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054277    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-sys-fs\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-script-lib\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054323 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vqpq\" (UniqueName: \"kubernetes.io/projected/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-kube-api-access-8vqpq\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054369    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwf77\" (UniqueName: \"kubernetes.io/projected/1e3c695e-3fac-49f4-838e-251b0c17fe67-kube-api-access-vwf77\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a304069c-7b5f-4ddf-9549-33797ac13658-agent-certs\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054462    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-var-lib-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054484    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cnibin\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cni-binary-copy\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-sys\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054578    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-slash\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-log-socket\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-bin\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-k8s-cni-cncf-io\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-kubelet-config\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-node-log\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.054710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th8x7\" (UniqueName: \"kubernetes.io/projected/91a3053e-bb6f-4678-b865-4bb0e24f7dad-kube-api-access-th8x7\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khb5q\" (UniqueName: \"kubernetes.io/projected/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-kube-api-access-khb5q\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-conf-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-lib-modules\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-device-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9j2g\" (UniqueName: \"kubernetes.io/projected/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kube-api-access-r9j2g\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a3053e-bb6f-4678-b865-4bb0e24f7dad-serviceca\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a304069c-7b5f-4ddf-9549-33797ac13658-konnectivity-ca\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-etc-kubernetes\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-var-lib-kubelet\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-netns\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-ovn\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e386bc-2d2b-438e-bb10-af8b05ea5b30-tmp-dir\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.054985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-registration-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-systemd-units\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055145    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysconfig\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.055440 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.055161    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-daemon-config\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055228    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-kubernetes\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.055281    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret podName:08444539-c0d4-4b26-9bbd-c75c21b32a21 nodeName:}" failed. No retries permitted until 2026-10-30 17:55:59.555226144 +0000 UTC m=+3.097837576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret") pod "global-pull-secret-syncer-trnpq" (UID: "08444539-c0d4-4b26-9bbd-c75c21b32a21") : object "kube-system"/"original-pull-secret" not registered
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-tmp\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-etc-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055368    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-kubernetes\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a3053e-bb6f-4678-b865-4bb0e24f7dad-host\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055429    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-var-lib-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-kubelet\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-modprobe-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055491    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cnibin\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055656    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-socket-dir-parent\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055719    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-hostroot\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055771    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-systemd\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055843    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-script-lib\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.056424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055861    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055914    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-multus\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.055958    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-multus-certs\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056076    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-socket-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056128    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cnibin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056247    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cni-binary-copy\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056327    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-conf\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056373    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056401    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056447    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056453    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056526    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-systemd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-host\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-iptables-alerter-script\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056662    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-config\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5424\" (UniqueName: \"kubernetes.io/projected/6fe540a2-725b-4723-bb8c-11b83afe9b2a-kube-api-access-p5424\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056757    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-host\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.057274 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056748    2576 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056805    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056830    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a3053e-bb6f-4678-b865-4bb0e24f7dad-serviceca\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056850    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-sys-fs\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056876    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-sys\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056905    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-slash\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056957    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-log-socket\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057014    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-bin\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057430    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-registration-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057498    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-netns\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057502    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-var-lib-kubelet\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057552    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-ovn\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057612    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-kubelet\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057616    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-etc-kubernetes\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057658    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-etc-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057709    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a3053e-bb6f-4678-b865-4bb0e24f7dad-host\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057853    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-k8s-cni-cncf-io\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057939    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-kubelet-config\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:59.058104 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057949    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-lib-modules\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.059138 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.057980    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-node-log\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.059138 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.058043    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-conf-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.059138 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.058265    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e386bc-2d2b-438e-bb10-af8b05ea5b30-tmp-dir\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:59.059138 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.058279    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-iptables-alerter-script\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:59.059138 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.058481    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.059138 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.058519    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-systemd-units\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.059138 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.056405    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-kubelet\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.059779 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.058723    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-device-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.060208 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.060180    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.060351 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.058749    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:55:59.060351 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.060294    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs podName:1e3c695e-3fac-49f4-838e-251b0c17fe67 nodeName:}" failed. No retries permitted until 2026-10-30 17:55:59.560266837 +0000 UTC m=+3.102878267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs") pod "network-metrics-daemon-dttj5" (UID: "1e3c695e-3fac-49f4-838e-251b0c17fe67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:55:59.061038 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061017    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-modprobe-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.061526 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061501    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a304069c-7b5f-4ddf-9549-33797ac13658-konnectivity-ca\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:55:59.061683 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061662    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.061771 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.061849 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hq7z\" (UniqueName: \"kubernetes.io/projected/04e386bc-2d2b-438e-bb10-af8b05ea5b30-kube-api-access-8hq7z\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:59.061849 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061819    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.061849 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061844    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-tmp\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.062091 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-run\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.062091 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-dbus\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:59.062091 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.061979    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-dbus\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:59.062091 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062034    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062090    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-run\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062144    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062277    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-netd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-env-overrides\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04e386bc-2d2b-438e-bb10-af8b05ea5b30-hosts-file\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062416    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-binary-copy\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-system-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062488    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-netns\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-bin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-host-slash\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.062601    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-tuned\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063092    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-netns\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063143    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-system-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063176    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-bin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063198    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-binary-copy\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063212    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-host-slash\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063250    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-netd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.065651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063320    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04e386bc-2d2b-438e-bb10-af8b05ea5b30-hosts-file\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:59.066614 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063399    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-daemon-config\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.066614 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063411    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a304069c-7b5f-4ddf-9549-33797ac13658-agent-certs\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:55:59.066614 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.063497    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-env-overrides\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.070243 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.070210    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c3da85c-aef5-4f3f-9381-8665d944c858-ovn-node-metrics-cert\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.070557 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.070537    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-tuned\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.085318 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.085286    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwf77\" (UniqueName: \"kubernetes.io/projected/1e3c695e-3fac-49f4-838e-251b0c17fe67-kube-api-access-vwf77\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:55:59.085527 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.085468    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc6lk\" (UniqueName: \"kubernetes.io/projected/0c3da85c-aef5-4f3f-9381-8665d944c858-kube-api-access-vc6lk\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.086193 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.086164    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lvb4\" (UniqueName: \"kubernetes.io/projected/dc9f7717-0e17-4ca2-aaa2-286a398e3435-kube-api-access-4lvb4\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.086466 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.086418    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5424\" (UniqueName: \"kubernetes.io/projected/6fe540a2-725b-4723-bb8c-11b83afe9b2a-kube-api-access-p5424\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.087156 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.087129    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th8x7\" (UniqueName: \"kubernetes.io/projected/91a3053e-bb6f-4678-b865-4bb0e24f7dad-kube-api-access-th8x7\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:59.087744 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.087723    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 17:55:59.087829 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.087755    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 17:55:59.087829 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.087771    2576 projected.go:196] Error preparing data for projected volume kube-api-access-nqhkd for pod openshift-network-diagnostics/network-check-target-d9ml4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:55:59.087934 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.087845    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd podName:ba85cd81-0f4d-4907-9190-8ba2e3199aa3 nodeName:}" failed. No retries permitted until 2026-10-30 17:55:59.587831224 +0000 UTC m=+3.130442639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nqhkd" (UniqueName: "kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd") pod "network-check-target-d9ml4" (UID: "ba85cd81-0f4d-4907-9190-8ba2e3199aa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:55:59.088626 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.088597    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9j2g\" (UniqueName: \"kubernetes.io/projected/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kube-api-access-r9j2g\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.090409 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.090385    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khb5q\" (UniqueName: \"kubernetes.io/projected/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-kube-api-access-khb5q\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.093795 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.093698    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hq7z\" (UniqueName: \"kubernetes.io/projected/04e386bc-2d2b-438e-bb10-af8b05ea5b30-kube-api-access-8hq7z\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:59.096489 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.096464    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vqpq\" (UniqueName: \"kubernetes.io/projected/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-kube-api-access-8vqpq\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:59.188316 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.188292    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7zqrb"
Oct 30 17:55:59.198997 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.198952    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:55:59.203192 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.202248    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9f7717_0e17_4ca2_aaa2_286a398e3435.slice/crio-90dddc2bf58d76d9a42310ecc46a91a129e4e43c7f28020fe20c3c12e311a499 WatchSource:0}: Error finding container 90dddc2bf58d76d9a42310ecc46a91a129e4e43c7f28020fe20c3c12e311a499: Status 404 returned error can't find the container with id 90dddc2bf58d76d9a42310ecc46a91a129e4e43c7f28020fe20c3c12e311a499
Oct 30 17:55:59.210222 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.210199    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:55:59.210753 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.210726    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe540a2_725b_4723_bb8c_11b83afe9b2a.slice/crio-050cd9d1ee8358e00f8f65c28318aba6f977b1ee2afad1122b2c727f7f057fab WatchSource:0}: Error finding container 050cd9d1ee8358e00f8f65c28318aba6f977b1ee2afad1122b2c727f7f057fab: Status 404 returned error can't find the container with id 050cd9d1ee8358e00f8f65c28318aba6f977b1ee2afad1122b2c727f7f057fab
Oct 30 17:55:59.220773 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.220748    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:55:59.221008 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.220983    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf557c0f3_8e66_416c_8c4b_cebb3fd60a25.slice/crio-3282e78544ee7ab35e0a2d9aaeb0daf14e087e263bdccee24698a52137f4d6ae WatchSource:0}: Error finding container 3282e78544ee7ab35e0a2d9aaeb0daf14e087e263bdccee24698a52137f4d6ae: Status 404 returned error can't find the container with id 3282e78544ee7ab35e0a2d9aaeb0daf14e087e263bdccee24698a52137f4d6ae
Oct 30 17:55:59.230144 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.230120    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:55:59.231011 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.230984    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda304069c_7b5f_4ddf_9549_33797ac13658.slice/crio-55b146394f9275007f459e946a522f4fe86e682c048c213a905141fb99fddc46 WatchSource:0}: Error finding container 55b146394f9275007f459e946a522f4fe86e682c048c213a905141fb99fddc46: Status 404 returned error can't find the container with id 55b146394f9275007f459e946a522f4fe86e682c048c213a905141fb99fddc46
Oct 30 17:55:59.238210 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.238072    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:55:59.241535 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.241511    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e386bc_2d2b_438e_bb10_af8b05ea5b30.slice/crio-892bff89c7902ee56649cc1734215455296c65b3256ddd266f83d468c6d1ce63 WatchSource:0}: Error finding container 892bff89c7902ee56649cc1734215455296c65b3256ddd266f83d468c6d1ce63: Status 404 returned error can't find the container with id 892bff89c7902ee56649cc1734215455296c65b3256ddd266f83d468c6d1ce63
Oct 30 17:55:59.248983 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.248955    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a3053e_bb6f_4678_b865_4bb0e24f7dad.slice/crio-fca71c4819366726fe0e60c275913c31b866a2c8e051c60a82d9cad082d8faf4 WatchSource:0}: Error finding container fca71c4819366726fe0e60c275913c31b866a2c8e051c60a82d9cad082d8faf4: Status 404 returned error can't find the container with id fca71c4819366726fe0e60c275913c31b866a2c8e051c60a82d9cad082d8faf4
Oct 30 17:55:59.260085 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.259862    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:55:59.271111 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.270910    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:55:59.272859 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.272819    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4947c1f_d461_4bd2_9cb0_4f7ff84b9ee3.slice/crio-a45417c32b7d6f944c419a810f2ee280f977b8c5d434aaf220b4e76eaea45b6c WatchSource:0}: Error finding container a45417c32b7d6f944c419a810f2ee280f977b8c5d434aaf220b4e76eaea45b6c: Status 404 returned error can't find the container with id a45417c32b7d6f944c419a810f2ee280f977b8c5d434aaf220b4e76eaea45b6c
Oct 30 17:55:59.283621 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.283587    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bc3bca_4b3f_4e7d_a14b_89228a7fc8ab.slice/crio-f9f5382ca9626b9391fd289f9b925080bd3059cb8255c9a462e0bdc1aacefb3e WatchSource:0}: Error finding container f9f5382ca9626b9391fd289f9b925080bd3059cb8255c9a462e0bdc1aacefb3e: Status 404 returned error can't find the container with id f9f5382ca9626b9391fd289f9b925080bd3059cb8255c9a462e0bdc1aacefb3e
Oct 30 17:55:59.284583 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.284563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:55:59.298283 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:55:59.298189    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c3da85c_aef5_4f3f_9381_8665d944c858.slice/crio-f5023e818d109e36e9626caf775d474710489b4424ef9a21e15aaf51aedef629 WatchSource:0}: Error finding container f5023e818d109e36e9626caf775d474710489b4424ef9a21e15aaf51aedef629: Status 404 returned error can't find the container with id f5023e818d109e36e9626caf775d474710489b4424ef9a21e15aaf51aedef629
Oct 30 17:55:59.566527 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.566350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:55:59.566527 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.566513    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 17:55:59.566754 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.566556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:55:59.566754 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.566579    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret podName:08444539-c0d4-4b26-9bbd-c75c21b32a21 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:00.566556125 +0000 UTC m=+4.109167539 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret") pod "global-pull-secret-syncer-trnpq" (UID: "08444539-c0d4-4b26-9bbd-c75c21b32a21") : object "kube-system"/"original-pull-secret" not registered
Oct 30 17:55:59.566754 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.566640    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:55:59.566754 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.566688    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs podName:1e3c695e-3fac-49f4-838e-251b0c17fe67 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:00.566672671 +0000 UTC m=+4.109284092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs") pod "network-metrics-daemon-dttj5" (UID: "1e3c695e-3fac-49f4-838e-251b0c17fe67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:55:59.668559 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.667734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:55:59.668559 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.667930    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 17:55:59.668559 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.667977    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 17:55:59.668559 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.667994    2576 projected.go:196] Error preparing data for projected volume kube-api-access-nqhkd for pod openshift-network-diagnostics/network-check-target-d9ml4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:55:59.668559 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:55:59.668057    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd podName:ba85cd81-0f4d-4907-9190-8ba2e3199aa3 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:00.668036789 +0000 UTC m=+4.210648226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqhkd" (UniqueName: "kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd") pod "network-check-target-d9ml4" (UID: "ba85cd81-0f4d-4907-9190-8ba2e3199aa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:55:59.908500 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.906554    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 17:50:59 +0000 UTC" deadline="2028-06-18 22:26:43.938075772 +0000 UTC"
Oct 30 17:55:59.908500 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.906615    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14332h30m44.031465015s"
Oct 30 17:55:59.934154 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.934068    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lxg8q" event={"ID":"04e386bc-2d2b-438e-bb10-af8b05ea5b30","Type":"ContainerStarted","Data":"892bff89c7902ee56649cc1734215455296c65b3256ddd266f83d468c6d1ce63"}
Oct 30 17:55:59.941262 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.941222    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s6687" event={"ID":"a304069c-7b5f-4ddf-9549-33797ac13658","Type":"ContainerStarted","Data":"55b146394f9275007f459e946a522f4fe86e682c048c213a905141fb99fddc46"}
Oct 30 17:55:59.952382 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.952344    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal" event={"ID":"167496dc01702046817db5cdac7ef75c","Type":"ContainerStarted","Data":"fca0eba0397f131174ac8108fb99fa0f85286f6053178a4c2980961e04326f1e"}
Oct 30 17:55:59.962640 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.958182    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk" event={"ID":"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab","Type":"ContainerStarted","Data":"f9f5382ca9626b9391fd289f9b925080bd3059cb8255c9a462e0bdc1aacefb3e"}
Oct 30 17:55:59.962640 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.961259    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerStarted","Data":"a45417c32b7d6f944c419a810f2ee280f977b8c5d434aaf220b4e76eaea45b6c"}
Oct 30 17:55:59.967349 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.967294    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6s62t" event={"ID":"f557c0f3-8e66-416c-8c4b-cebb3fd60a25","Type":"ContainerStarted","Data":"3282e78544ee7ab35e0a2d9aaeb0daf14e087e263bdccee24698a52137f4d6ae"}
Oct 30 17:55:59.975326 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.975297    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hmlkj" event={"ID":"6fe540a2-725b-4723-bb8c-11b83afe9b2a","Type":"ContainerStarted","Data":"050cd9d1ee8358e00f8f65c28318aba6f977b1ee2afad1122b2c727f7f057fab"}
Oct 30 17:55:59.988680 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.988647    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7zqrb" event={"ID":"dc9f7717-0e17-4ca2-aaa2-286a398e3435","Type":"ContainerStarted","Data":"90dddc2bf58d76d9a42310ecc46a91a129e4e43c7f28020fe20c3c12e311a499"}
Oct 30 17:55:59.993806 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.993773    2576 generic.go:356] "Generic (PLEG): container finished" podID="8158fd6db0c3186f36b1583fbfe4e93d" containerID="580ea1c67f42aded753ef10d011b6fb5a5503c09d3b20688b555dc4c96367f7f" exitCode=0
Oct 30 17:55:59.993922 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.993859    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal" event={"ID":"8158fd6db0c3186f36b1583fbfe4e93d","Type":"ContainerDied","Data":"580ea1c67f42aded753ef10d011b6fb5a5503c09d3b20688b555dc4c96367f7f"}
Oct 30 17:55:59.996930 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.996894    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"f5023e818d109e36e9626caf775d474710489b4424ef9a21e15aaf51aedef629"}
Oct 30 17:56:00.000665 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:55:59.999869    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vxvtw" event={"ID":"91a3053e-bb6f-4678-b865-4bb0e24f7dad","Type":"ContainerStarted","Data":"fca71c4819366726fe0e60c275913c31b866a2c8e051c60a82d9cad082d8faf4"}
Oct 30 17:56:00.050599 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.050537    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal" podStartSLOduration=1.050515195 podStartE2EDuration="1.050515195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:00.002159053 +0000 UTC m=+3.544770494" watchObservedRunningTime="2026-10-30 17:56:00.050515195 +0000 UTC m=+3.593126635"
Oct 30 17:56:00.577536 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.577440    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:00.577536 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.577511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:00.577786 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.577698    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:00.577786 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.577761    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs podName:1e3c695e-3fac-49f4-838e-251b0c17fe67 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:02.5777421 +0000 UTC m=+6.120353531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs") pod "network-metrics-daemon-dttj5" (UID: "1e3c695e-3fac-49f4-838e-251b0c17fe67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:00.578219 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.578196    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:00.578324 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.578256    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret podName:08444539-c0d4-4b26-9bbd-c75c21b32a21 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:02.578239682 +0000 UTC m=+6.120851104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret") pod "global-pull-secret-syncer-trnpq" (UID: "08444539-c0d4-4b26-9bbd-c75c21b32a21") : object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:00.679109 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.678941    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:00.679250 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.679193    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 17:56:00.679250 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.679218    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 17:56:00.679250 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.679232    2576 projected.go:196] Error preparing data for projected volume kube-api-access-nqhkd for pod openshift-network-diagnostics/network-check-target-d9ml4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:00.679438 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.679291    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd podName:ba85cd81-0f4d-4907-9190-8ba2e3199aa3 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:02.679272675 +0000 UTC m=+6.221884098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqhkd" (UniqueName: "kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd") pod "network-check-target-d9ml4" (UID: "ba85cd81-0f4d-4907-9190-8ba2e3199aa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:00.907569 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.907521    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 17:50:59 +0000 UTC" deadline="2028-05-13 17:28:54.253221519 +0000 UTC"
Oct 30 17:56:00.907569 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.907566    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13463h32m53.345660411s"
Oct 30 17:56:00.913082 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.913048    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:00.913550 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.913211    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:00.913683 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.913662    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:00.913793 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.913772    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:00.913856 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:00.913840    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:00.913932 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:00.913912    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:01.018059 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:01.018022    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal" event={"ID":"8158fd6db0c3186f36b1583fbfe4e93d","Type":"ContainerStarted","Data":"e16eccafccea1c053ce0317a6afb0f27fc643a675579860ac8964bb295f973aa"}
Oct 30 17:56:02.040152 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:02.040049    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerStarted","Data":"f72a7876e6b80ffe3d1675c304d9b956040506ab8cc6afc6ac61f751117112ba"}
Oct 30 17:56:02.129333 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:02.129257    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal" podStartSLOduration=3.129215528 podStartE2EDuration="3.129215528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:01.06780074 +0000 UTC m=+4.610412182" watchObservedRunningTime="2026-10-30 17:56:02.129215528 +0000 UTC m=+5.671826970"
Oct 30 17:56:02.594310 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:02.594271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:02.594489 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:02.594354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:02.594489 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.594469    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:02.594628 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.594545    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret podName:08444539-c0d4-4b26-9bbd-c75c21b32a21 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:06.594525679 +0000 UTC m=+10.137137113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret") pod "global-pull-secret-syncer-trnpq" (UID: "08444539-c0d4-4b26-9bbd-c75c21b32a21") : object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:02.595010 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.594966    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:02.595137 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.595046    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs podName:1e3c695e-3fac-49f4-838e-251b0c17fe67 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:06.595029612 +0000 UTC m=+10.137641043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs") pod "network-metrics-daemon-dttj5" (UID: "1e3c695e-3fac-49f4-838e-251b0c17fe67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:02.695408 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:02.695361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:02.695639 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.695620    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 17:56:02.695715 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.695651    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 17:56:02.695715 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.695665    2576 projected.go:196] Error preparing data for projected volume kube-api-access-nqhkd for pod openshift-network-diagnostics/network-check-target-d9ml4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:02.695834 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.695728    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd podName:ba85cd81-0f4d-4907-9190-8ba2e3199aa3 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:06.69570662 +0000 UTC m=+10.238318080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqhkd" (UniqueName: "kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd") pod "network-check-target-d9ml4" (UID: "ba85cd81-0f4d-4907-9190-8ba2e3199aa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:02.917447 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:02.916957    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:02.917447 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:02.917095    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:02.917447 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.917200    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:02.917447 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.917097    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:02.917447 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:02.917275    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:02.917447 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:02.917359    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:03.050351 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:03.049362    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="f72a7876e6b80ffe3d1675c304d9b956040506ab8cc6afc6ac61f751117112ba" exitCode=0
Oct 30 17:56:03.050351 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:03.049413    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerDied","Data":"f72a7876e6b80ffe3d1675c304d9b956040506ab8cc6afc6ac61f751117112ba"}
Oct 30 17:56:04.059777 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:04.059076    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s6687" event={"ID":"a304069c-7b5f-4ddf-9549-33797ac13658","Type":"ContainerStarted","Data":"b499b85f2243c38925a66d11e03d3471b73d85ac7e2223447e6f8a27fcb24ad6"}
Oct 30 17:56:04.452302 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:04.452100    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:56:04.452952 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:04.452744    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:56:04.503113 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:04.502431    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s6687" podStartSLOduration=3.391589571 podStartE2EDuration="7.502408532s" totalImagesPullingTime="4.110818961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:04.109612615 +0000 UTC m=+7.652224056" watchObservedRunningTime="2026-10-30 17:56:04.502408532 +0000 UTC m=+8.045020009"
Oct 30 17:56:04.919543 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:04.918452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:04.919543 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:04.918560    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:04.919543 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:04.919250    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:04.919543 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:04.919349    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:04.919543 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:04.919410    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:04.919543 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:04.919479    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:05.072648 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:05.072572    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vxvtw" event={"ID":"91a3053e-bb6f-4678-b865-4bb0e24f7dad","Type":"ContainerStarted","Data":"d3a8c39319a99a4ddab3c0e672164e71a7db8a38f42b718e9403000b98ae3d0e"}
Oct 30 17:56:05.119543 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:05.119404    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vxvtw" podStartSLOduration=3.134680059 podStartE2EDuration="8.119383032s" totalImagesPullingTime="4.984702973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:05.118996405 +0000 UTC m=+8.661607845" watchObservedRunningTime="2026-10-30 17:56:05.119383032 +0000 UTC m=+8.661994472"
Oct 30 17:56:06.080392 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.080355    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk" event={"ID":"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab","Type":"ContainerStarted","Data":"226a0e6322d713aa1b9201ac62aaf7b0f5554a8181602d8c0400854f7f4e703a"}
Oct 30 17:56:06.080822 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.080463    2576 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 17:56:06.636341 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.636292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:06.636526 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.636401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:06.636598 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.636529    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:06.636652 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.636596    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret podName:08444539-c0d4-4b26-9bbd-c75c21b32a21 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:14.636575686 +0000 UTC m=+18.179187102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret") pod "global-pull-secret-syncer-trnpq" (UID: "08444539-c0d4-4b26-9bbd-c75c21b32a21") : object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:06.637102 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.637075    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:06.637194 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.637146    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs podName:1e3c695e-3fac-49f4-838e-251b0c17fe67 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:14.6371281 +0000 UTC m=+18.179739520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs") pod "network-metrics-daemon-dttj5" (UID: "1e3c695e-3fac-49f4-838e-251b0c17fe67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:06.736964 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.736922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:06.737146 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.737118    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 17:56:06.737146 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.737139    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 17:56:06.737253 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.737152    2576 projected.go:196] Error preparing data for projected volume kube-api-access-nqhkd for pod openshift-network-diagnostics/network-check-target-d9ml4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:06.737253 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.737213    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd podName:ba85cd81-0f4d-4907-9190-8ba2e3199aa3 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:14.737196025 +0000 UTC m=+18.279807461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqhkd" (UniqueName: "kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd") pod "network-check-target-d9ml4" (UID: "ba85cd81-0f4d-4907-9190-8ba2e3199aa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:06.913863 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.913837    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:06.914020 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.913960    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:06.914319 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.914301    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:06.914452 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.914390    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:06.914578 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.914450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:06.914578 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:06.914533    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:06.988586 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:06.988340    2576 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 17:56:07.087760 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:07.087717    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk" event={"ID":"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab","Type":"ContainerStarted","Data":"0f9d0513ec530e95809fb083b7f54a484ebf7897b7b00bc38ccaad9494b822ef"}
Oct 30 17:56:07.913584 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:07.913273    2576 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T17:56:06.988356279Z","UUID":"49ba24f8-8762-4a32-9630-3abad627074a","Handler":null,"Name":"","Endpoint":""}
Oct 30 17:56:07.914989 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:07.914962    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 17:56:07.914989 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:07.914994    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 17:56:08.092118 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.092082    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lxg8q" event={"ID":"04e386bc-2d2b-438e-bb10-af8b05ea5b30","Type":"ContainerStarted","Data":"9cd7c668a4461b570ab1277b2589b345aa32bdc15bbba18ca7509268f334b740"}
Oct 30 17:56:08.094629 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.094598    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk" event={"ID":"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab","Type":"ContainerStarted","Data":"e0cafb893013f39cfdd647519e2a8c4273f46b2ba9155e161459a84932fb78af"}
Oct 30 17:56:08.096704 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.096666    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="924cbf10b3686ae01735a70cc88c5ab217549ea641730075743ac07fb1915e84" exitCode=0
Oct 30 17:56:08.096821 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.096752    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerDied","Data":"924cbf10b3686ae01735a70cc88c5ab217549ea641730075743ac07fb1915e84"}
Oct 30 17:56:08.098381 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.098356    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7zqrb" event={"ID":"dc9f7717-0e17-4ca2-aaa2-286a398e3435","Type":"ContainerStarted","Data":"752ade8990fa743dbcc5933122c1bf83fe326d8502a55495eeddf294dd37e5e9"}
Oct 30 17:56:08.140005 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.139880    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-lxg8q" podStartSLOduration=3.157776578 podStartE2EDuration="11.139862028s" totalImagesPullingTime="7.98208545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:08.136381971 +0000 UTC m=+11.678993399" watchObservedRunningTime="2026-10-30 17:56:08.139862028 +0000 UTC m=+11.682473465"
Oct 30 17:56:08.239810 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.238775    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk" podStartSLOduration=2.972453667 podStartE2EDuration="11.238757351s" totalImagesPullingTime="8.266303684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:08.23864299 +0000 UTC m=+11.781254426" watchObservedRunningTime="2026-10-30 17:56:08.238757351 +0000 UTC m=+11.781368788"
Oct 30 17:56:08.286062 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.286016    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-7zqrb" podStartSLOduration=3.2774852709999998 podStartE2EDuration="11.286005897s" totalImagesPullingTime="8.008520626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:08.285804044 +0000 UTC m=+11.828415480" watchObservedRunningTime="2026-10-30 17:56:08.286005897 +0000 UTC m=+11.828617334"
Oct 30 17:56:08.912566 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.912537    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:08.912566 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.912557    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:08.912749 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:08.912672    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:08.912749 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:08.912686    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:08.912812 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:08.912779    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:08.912977 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:08.912883    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:09.103641 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:09.103603    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6s62t" event={"ID":"f557c0f3-8e66-416c-8c4b-cebb3fd60a25","Type":"ContainerStarted","Data":"a6163b0891aadd5620de74fc37c24967af8d56fbab4bb66f3e868b2eedaef900"}
Oct 30 17:56:09.148416 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:09.148362    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6s62t" podStartSLOduration=4.00077755 podStartE2EDuration="12.148348664s" totalImagesPullingTime="8.147571114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:09.148010719 +0000 UTC m=+12.690622154" watchObservedRunningTime="2026-10-30 17:56:09.148348664 +0000 UTC m=+12.690960101"
Oct 30 17:56:10.107617 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:10.107579    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="69b3ccc672b3f1486f7610dedae2ab1694b19d2e531f5d0b0439e86836c91916" exitCode=0
Oct 30 17:56:10.108383 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:10.107654    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerDied","Data":"69b3ccc672b3f1486f7610dedae2ab1694b19d2e531f5d0b0439e86836c91916"}
Oct 30 17:56:10.109788 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:10.109769    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hmlkj" event={"ID":"6fe540a2-725b-4723-bb8c-11b83afe9b2a","Type":"ContainerStarted","Data":"862c196e1e8f84a69d83203d5ddc4f74ac932d544811c9d72232bfbf5e102118"}
Oct 30 17:56:10.237415 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:10.237350    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hmlkj" podStartSLOduration=2.816773726 podStartE2EDuration="13.237333651s" totalImagesPullingTime="10.420559925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:10.236692094 +0000 UTC m=+13.779303531" watchObservedRunningTime="2026-10-30 17:56:10.237333651 +0000 UTC m=+13.779945088"
Oct 30 17:56:10.913178 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:10.912949    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:10.913359 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:10.913306    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:10.913359 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:10.912998    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:10.913463 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:10.913418    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:10.913463 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:10.912960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:10.913553 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:10.913508    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:12.116580 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:12.116537    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="64d8d09702db5b9a5592ef8cb793db6595ddef1b8d27b86426a53f2b35be981f" exitCode=0
Oct 30 17:56:12.117094 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:12.116603    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerDied","Data":"64d8d09702db5b9a5592ef8cb793db6595ddef1b8d27b86426a53f2b35be981f"}
Oct 30 17:56:12.912338 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:12.912305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:12.912541 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:12.912436    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:12.912541 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:12.912305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:12.912666 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:12.912553    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:12.912666 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:12.912601    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:12.912666 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:12.912663    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:14.699305 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:14.699124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:14.699648 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:14.699333    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:14.699648 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.699272    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:14.699648 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.699429    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:14.699648 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.699439    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret podName:08444539-c0d4-4b26-9bbd-c75c21b32a21 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:30.69942204 +0000 UTC m=+34.242033458 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret") pod "global-pull-secret-syncer-trnpq" (UID: "08444539-c0d4-4b26-9bbd-c75c21b32a21") : object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:14.699648 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.699467    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs podName:1e3c695e-3fac-49f4-838e-251b0c17fe67 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:30.699454388 +0000 UTC m=+34.242065802 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs") pod "network-metrics-daemon-dttj5" (UID: "1e3c695e-3fac-49f4-838e-251b0c17fe67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:14.800097 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:14.800064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:14.800267 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.800188    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 17:56:14.800267 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.800201    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 17:56:14.800267 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.800210    2576 projected.go:196] Error preparing data for projected volume kube-api-access-nqhkd for pod openshift-network-diagnostics/network-check-target-d9ml4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:14.800267 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.800253    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd podName:ba85cd81-0f4d-4907-9190-8ba2e3199aa3 nodeName:}" failed. No retries permitted until 2026-10-30 17:56:30.800240328 +0000 UTC m=+34.342851742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqhkd" (UniqueName: "kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd") pod "network-check-target-d9ml4" (UID: "ba85cd81-0f4d-4907-9190-8ba2e3199aa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:14.913299 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:14.913265    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:14.913299 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:14.913295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:14.913538 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:14.913265    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:14.913538 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.913381    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:14.913538 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.913471    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:14.913681 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:14.913588    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:15.125574 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:15.125533    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="ac3f698457feeb271bc772ec1c42995c71de66f9b8c4229290143410044e7e3b" exitCode=0
Oct 30 17:56:15.125717 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:15.125653    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerDied","Data":"ac3f698457feeb271bc772ec1c42995c71de66f9b8c4229290143410044e7e3b"}
Oct 30 17:56:15.128602 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:15.128585    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"8f1eb9b4afd1ae4be8c16f4579109edd3ddf49741f6455f491efbdd4aa4dcb09"}
Oct 30 17:56:15.128651 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:15.128644    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"25916e57f2fa39210121747bcb398d6b61c4ef86386e1edabae5be8eaf8b0305"}
Oct 30 17:56:15.128688 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:15.128677    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"403d35bf73033a4f976662f638fcdf1b88ad729fe78834aca7175056a1caaf75"}
Oct 30 17:56:15.128719 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:15.128705    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"07b3485c25f41acb6086c593f938507e3084e27400ee71504d2aac8245a2a3aa"}
Oct 30 17:56:15.128759 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:15.128734    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"6e567767729ba1b2eed7bb10686aa74c5bc1e942cc9a41f2fde6bef62e49edff"}
Oct 30 17:56:15.128789 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:15.128763    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"03bd87006e201bc4c65505c6eb7209e73aaeaea66c274560ad2e352978d0853e"}
Oct 30 17:56:16.135456 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:16.135420    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="e946c081458c1ce0cbe6c9d8af6ccfb16ad9506ebb43cd622b76d2bcfff69a85" exitCode=0
Oct 30 17:56:16.135996 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:16.135463    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerDied","Data":"e946c081458c1ce0cbe6c9d8af6ccfb16ad9506ebb43cd622b76d2bcfff69a85"}
Oct 30 17:56:16.913333 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:16.913253    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:16.913474 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:16.913381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:16.913474 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:16.913423    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:16.913474 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:16.913432    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:16.913626 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:16.913504    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:16.913735 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:16.913616    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:17.142965 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:17.142898    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"2b4879b0e61bc5bd81239cc73efd6511dbadcedbcc1b81caa0908494b4bf25cb"}
Oct 30 17:56:17.146036 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:17.146012    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" event={"ID":"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3","Type":"ContainerStarted","Data":"ee2b70a987785a8570dccddfe91d736b7fb85fc6c4f32c2d69741aa2cc24c8ff"}
Oct 30 17:56:17.198701 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:17.198624    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lfmxr" podStartSLOduration=9.437262237 podStartE2EDuration="20.198611681s" totalImagesPullingTime="10.761349444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:17.198268624 +0000 UTC m=+20.740880061" watchObservedRunningTime="2026-10-30 17:56:17.198611681 +0000 UTC m=+20.741223116"
Oct 30 17:56:18.913069 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:18.913035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:18.913630 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:18.913072    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:18.913630 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:18.913150    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:18.913630 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:18.913220    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:18.913630 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:18.913303    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:18.913630 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:18.913385    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:19.153947 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:19.153902    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" event={"ID":"0c3da85c-aef5-4f3f-9381-8665d944c858","Type":"ContainerStarted","Data":"1042bdbb777d710745ad04375f682a85a849994c30cc7166f3144796335135fe"}
Oct 30 17:56:19.154213 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:19.154193    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:56:19.154297 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:19.154247    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:56:19.170004 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:19.169830    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:56:19.258064 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:19.258021    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg" podStartSLOduration=7.236087451 podStartE2EDuration="22.258008361s" totalImagesPullingTime="15.02192091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:19.257699872 +0000 UTC m=+22.800311330" watchObservedRunningTime="2026-10-30 17:56:19.258008361 +0000 UTC m=+22.800619797"
Oct 30 17:56:20.156460 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:20.156428    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:56:20.172236 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:20.172211    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:56:20.839965 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:20.839873    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d9ml4"]
Oct 30 17:56:20.840139 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:20.840057    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:20.840216 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:20.840152    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:20.840526 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:20.840503    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-trnpq"]
Oct 30 17:56:20.840623 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:20.840610    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:20.840706 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:20.840689    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:20.842014 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:20.841996    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dttj5"]
Oct 30 17:56:20.842089 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:20.842075    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:20.842162 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:20.842146    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:22.912758 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:22.912726    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:22.913234 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:22.912727    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:22.913234 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:22.912874    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:22.913234 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:22.912889    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:22.913234 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:22.913024    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:22.913234 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:22.913095    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:24.912449 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:24.912415    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:24.912449 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:24.912451    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:24.912996 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:24.912428    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:24.912996 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:24.912539    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:24.912996 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:24.912586    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:24.912996 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:24.912671    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:25.343248 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:25.343180    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxg8q_04e386bc-2d2b-438e-bb10-af8b05ea5b30/dns-node-resolver/0.log"
Oct 30 17:56:26.276326 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:26.276295    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vxvtw_91a3053e-bb6f-4678-b865-4bb0e24f7dad/node-ca/0.log"
Oct 30 17:56:26.913368 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:26.913341    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:26.913543 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:26.913422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:26.913543 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:26.913455    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:26.913543 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:26.913498    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:26.913664 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:26.913552    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:26.913664 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:26.913611    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:27.408444 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:27.408409    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:56:27.408836 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:27.408544    2576 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 17:56:27.409005 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:27.408992    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:56:28.912817 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:28.912784    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:28.913340 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:28.912817    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:28.913340 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:28.912897    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:28.913340 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:28.912968    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:28.913340 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:28.913084    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:28.913340 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:28.913166    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:30.711877 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:30.711845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:30.712285 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:30.711887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:30.712285 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.711995    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:30.712285 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.711997    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:30.712285 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.712051    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs podName:1e3c695e-3fac-49f4-838e-251b0c17fe67 nodeName:}" failed. No retries permitted until 2026-10-30 17:57:02.712032703 +0000 UTC m=+66.254644135 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs") pod "network-metrics-daemon-dttj5" (UID: "1e3c695e-3fac-49f4-838e-251b0c17fe67") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 17:56:30.712285 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.712068    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret podName:08444539-c0d4-4b26-9bbd-c75c21b32a21 nodeName:}" failed. No retries permitted until 2026-10-30 17:57:02.71205971 +0000 UTC m=+66.254671125 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret") pod "global-pull-secret-syncer-trnpq" (UID: "08444539-c0d4-4b26-9bbd-c75c21b32a21") : object "kube-system"/"original-pull-secret" not registered
Oct 30 17:56:30.812338 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:30.812308    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:30.812489 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.812437    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 17:56:30.812489 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.812459    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 17:56:30.812489 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.812470    2576 projected.go:196] Error preparing data for projected volume kube-api-access-nqhkd for pod openshift-network-diagnostics/network-check-target-d9ml4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:30.812649 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.812525    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd podName:ba85cd81-0f4d-4907-9190-8ba2e3199aa3 nodeName:}" failed. No retries permitted until 2026-10-30 17:57:02.812508364 +0000 UTC m=+66.355119798 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqhkd" (UniqueName: "kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd") pod "network-check-target-d9ml4" (UID: "ba85cd81-0f4d-4907-9190-8ba2e3199aa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 17:56:30.912599 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:30.912573    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:30.912599 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:30.912588    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:30.912767 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:30.912587    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:30.912767 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.912698    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:30.912843 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.912763    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:30.912879 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:30.912834    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:32.913356 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:32.913328    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:32.913356 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:32.913356    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:32.913787 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:32.913424    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:32.913787 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:32.913440    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:32.913787 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:32.913527    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:32.913787 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:32.913572    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:34.912980 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:34.912945    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:34.913444 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:34.912946    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:34.913444 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:34.913070    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:34.913444 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:34.913065    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:34.913444 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:34.913153    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:34.913444 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:34.913228    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:36.913606 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:36.913573    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:36.914117 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:36.913674    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:36.914117 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:36.913714    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:36.914117 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:36.913729    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:36.914117 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:36.913801    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:36.914117 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:36.913887    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:38.912509 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:38.912483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:38.912847 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:38.912483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:38.912847 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:38.912582    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:38.912847 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:38.912492    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:38.912847 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:38.912666    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:38.912847 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:38.912738    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:40.912459 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:40.912425    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:40.912960 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:40.912484    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:40.912960 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:40.912578    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:40.912960 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:40.912576    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:40.912960 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:40.912708    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:40.912960 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:40.912758    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:42.913065 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:42.913035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:42.913460 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:42.913035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:42.913460 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:42.913131    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:42.913460 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:42.913035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:42.913460 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:42.913245    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:42.913460 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:42.913288    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:44.913150 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:44.913119    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:44.913150 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:44.913134    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:44.913671 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:44.913144    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:44.913671 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:44.913255    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:44.913671 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:44.913357    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:44.913671 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:44.913435    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:46.915574 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:46.915545    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:46.916063 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:46.915666    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d9ml4" podUID="ba85cd81-0f4d-4907-9190-8ba2e3199aa3"
Oct 30 17:56:46.916063 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:46.915685    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:46.916063 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:46.915711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:46.916063 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:46.915784    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trnpq" podUID="08444539-c0d4-4b26-9bbd-c75c21b32a21"
Oct 30 17:56:46.916063 ip-10-0-131-31 kubenswrapper[2576]: E1030 17:56:46.915870    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dttj5" podUID="1e3c695e-3fac-49f4-838e-251b0c17fe67"
Oct 30 17:56:47.208778 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.208747    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeReady"
Oct 30 17:56:47.208963 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.208879    2576 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 17:56:47.286855 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.286827    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zwh9d"]
Oct 30 17:56:47.288638 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.288622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.290430 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.290413    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2ljs5"]
Oct 30 17:56:47.291853 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.291840    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.310797 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.310773    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 17:56:47.310896 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.310807    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 17:56:47.310896 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.310843    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 17:56:47.310896 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.310883    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 17:56:47.311074 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.310951    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2dl6z\""
Oct 30 17:56:47.311587 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.311566    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zwh9d"]
Oct 30 17:56:47.314043 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.314026    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 17:56:47.315255 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.315237    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2ljs5"]
Oct 30 17:56:47.329122 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329101    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9zl\" (UniqueName: \"kubernetes.io/projected/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-api-access-fm9zl\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.329206 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-data-volume\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.329206 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329150    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x42gn\" (UniqueName: \"kubernetes.io/projected/a6ee33fb-66e9-4483-b3c1-a289e9620de4-kube-api-access-x42gn\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.329206 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6ee33fb-66e9-4483-b3c1-a289e9620de4-metrics-tls\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.329302 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee33fb-66e9-4483-b3c1-a289e9620de4-tmp-dir\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.329333 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ee33fb-66e9-4483-b3c1-a289e9620de4-config-volume\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.329333 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-crio-socket\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.329389 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.329389 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.329361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.383952 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.383902    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lvffz"]
Oct 30 17:56:47.385594 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.385576    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:56:47.408883 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.408861    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lvffz"]
Oct 30 17:56:47.411222 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.411205    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 17:56:47.411396 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.411381    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 17:56:47.411494 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.411475    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksmnn\""
Oct 30 17:56:47.411552 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.411504    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7j6rv\""
Oct 30 17:56:47.411552 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.411528    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 17:56:47.411552 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.411532    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 17:56:47.429804 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.429786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6ee33fb-66e9-4483-b3c1-a289e9620de4-metrics-tls\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.429898 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.429812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee33fb-66e9-4483-b3c1-a289e9620de4-tmp-dir\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.429898 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.429855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ee33fb-66e9-4483-b3c1-a289e9620de4-config-volume\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.429898 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.429882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-crio-socket\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.430050 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.429932    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.430050 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.429962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.430050 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.429988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-cert\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:56:47.430188 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430110    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-crio-socket\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.430241 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430209    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdd5b\" (UniqueName: \"kubernetes.io/projected/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-kube-api-access-fdd5b\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:56:47.430241 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430216    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee33fb-66e9-4483-b3c1-a289e9620de4-tmp-dir\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.430340 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9zl\" (UniqueName: \"kubernetes.io/projected/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-api-access-fm9zl\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.430392 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-data-volume\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.430392 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x42gn\" (UniqueName: \"kubernetes.io/projected/a6ee33fb-66e9-4483-b3c1-a289e9620de4-kube-api-access-x42gn\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.430697 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430671    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ee33fb-66e9-4483-b3c1-a289e9620de4-config-volume\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.430822 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430797    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-data-volume\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.430939 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.430700    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.433735 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.433713    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6ee33fb-66e9-4483-b3c1-a289e9620de4-metrics-tls\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.433816 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.433719    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.455736 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.455711    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9zl\" (UniqueName: \"kubernetes.io/projected/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-api-access-fm9zl\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.455852 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.455791    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x42gn\" (UniqueName: \"kubernetes.io/projected/a6ee33fb-66e9-4483-b3c1-a289e9620de4-kube-api-access-x42gn\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.531672 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.531605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-cert\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:56:47.531672 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.531638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdd5b\" (UniqueName: \"kubernetes.io/projected/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-kube-api-access-fdd5b\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:56:47.534160 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.534137    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-cert\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:56:47.557517 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.557488    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdd5b\" (UniqueName: \"kubernetes.io/projected/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-kube-api-access-fdd5b\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:56:47.602339 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.602310    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:56:47.605702 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.605681    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:47.694606 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.694582    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:56:47.929341 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.929314    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2ljs5"]
Oct 30 17:56:47.932936 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.932069    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zwh9d"]
Oct 30 17:56:47.933646 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:56:47.933621    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ee33fb_66e9_4483_b3c1_a289e9620de4.slice/crio-04353ae0bb37b5944719e4306001e51e44a5b7fa7619139a861eb0146f6b33ab WatchSource:0}: Error finding container 04353ae0bb37b5944719e4306001e51e44a5b7fa7619139a861eb0146f6b33ab: Status 404 returned error can't find the container with id 04353ae0bb37b5944719e4306001e51e44a5b7fa7619139a861eb0146f6b33ab
Oct 30 17:56:47.934701 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:56:47.934678    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffa15d5_c851_4b6e_bfc7_faa42f332f0b.slice/crio-762289fdc4e13abdfc08e25ba85382d613aa87b23eccdcd31cd44b7e51aebc9d WatchSource:0}: Error finding container 762289fdc4e13abdfc08e25ba85382d613aa87b23eccdcd31cd44b7e51aebc9d: Status 404 returned error can't find the container with id 762289fdc4e13abdfc08e25ba85382d613aa87b23eccdcd31cd44b7e51aebc9d
Oct 30 17:56:47.988874 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:47.988845    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lvffz"]
Oct 30 17:56:48.210261 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.210175    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ljs5" event={"ID":"a6ee33fb-66e9-4483-b3c1-a289e9620de4","Type":"ContainerStarted","Data":"04353ae0bb37b5944719e4306001e51e44a5b7fa7619139a861eb0146f6b33ab"}
Oct 30 17:56:48.211723 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.211697    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwh9d" event={"ID":"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b","Type":"ContainerStarted","Data":"b644285694f4da5fd8fbdda7b52f29ab0d30b4df7f841f6cb54cc5009a6b336c"}
Oct 30 17:56:48.211827 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.211755    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwh9d" event={"ID":"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b","Type":"ContainerStarted","Data":"762289fdc4e13abdfc08e25ba85382d613aa87b23eccdcd31cd44b7e51aebc9d"}
Oct 30 17:56:48.212853 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.212831    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lvffz" event={"ID":"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f","Type":"ContainerStarted","Data":"74f8ab7cef4403c5e87a4457a33a5b13030373364ae5a1b94b0ecb34a6755af8"}
Oct 30 17:56:48.917253 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.917222    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:56:48.917253 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.917248    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:56:48.917653 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.917633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:56:48.935630 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.935610    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 17:56:48.935990 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.935767    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 17:56:48.950408 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.950371    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bkbt2\""
Oct 30 17:56:48.950525 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.950422    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 17:56:48.950525 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.950457    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dxtgs\""
Oct 30 17:56:48.950525 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.950371    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-tzlmk\""
Oct 30 17:56:48.950525 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:48.950490    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 17:56:49.219138 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:49.219057    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwh9d" event={"ID":"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b","Type":"ContainerStarted","Data":"3d83d05856ca40b524b6715e80202c2d18672d57c95f6615c593ff44187926cf"}
Oct 30 17:56:50.224353 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.224321    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ljs5" event={"ID":"a6ee33fb-66e9-4483-b3c1-a289e9620de4","Type":"ContainerStarted","Data":"09c0c71fbb2f0257d8b3fdadb7beae4d81ef8750947967c746480ba3f03ba89c"}
Oct 30 17:56:50.224751 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.224394    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ljs5" event={"ID":"a6ee33fb-66e9-4483-b3c1-a289e9620de4","Type":"ContainerStarted","Data":"3ebf454539ca702aacebfc3191c6ad83e0accce68dd70b4f803ee1de6e026792"}
Oct 30 17:56:50.224751 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.224476    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:56:50.226831 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.226811    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zwh9d" event={"ID":"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b","Type":"ContainerStarted","Data":"ec94ebbca172f65e99f2701f26ca4ac9ee02d8230c5ebad2eacf0bd74f0ced6e"}
Oct 30 17:56:50.278426 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.278373    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-2ljs5" podStartSLOduration=1.4909563239999999 podStartE2EDuration="3.27835652s" totalImagesPullingTime="1.787400196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 17:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:50.277550633 +0000 UTC m=+53.820162070" watchObservedRunningTime="2026-10-30 17:56:50.27835652 +0000 UTC m=+53.820967957"
Oct 30 17:56:50.325266 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.325194    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zwh9d" podStartSLOduration=1.243859899 podStartE2EDuration="3.325177861s" totalImagesPullingTime="2.081317962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 17:56:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:50.324450815 +0000 UTC m=+53.867062252" watchObservedRunningTime="2026-10-30 17:56:50.325177861 +0000 UTC m=+53.867789297"
Oct 30 17:56:50.383160 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.383134    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2ljs5_a6ee33fb-66e9-4483-b3c1-a289e9620de4/kube-rbac-proxy/0.log"
Oct 30 17:56:50.438316 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.438231    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"]
Oct 30 17:56:50.440161 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.440145    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.460173 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.459883    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 17:56:50.460337 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.460320    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 17:56:50.460450 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.460436    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 17:56:50.460554 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.460540    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-98z5p\""
Oct 30 17:56:50.460616 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.460551    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 17:56:50.460616 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.460600    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 17:56:50.461617 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.461594    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"]
Oct 30 17:56:50.569062 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.569024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.569062 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.569068    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ca9cc19-6578-4e8d-a424-61ed63f9b562-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.569283 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.569253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.569346 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.569302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jw5v\" (UniqueName: \"kubernetes.io/projected/9ca9cc19-6578-4e8d-a424-61ed63f9b562-kube-api-access-2jw5v\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.670382 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.670347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.670382 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.670386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jw5v\" (UniqueName: \"kubernetes.io/projected/9ca9cc19-6578-4e8d-a424-61ed63f9b562-kube-api-access-2jw5v\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.670610 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.670422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.670610 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.670441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ca9cc19-6578-4e8d-a424-61ed63f9b562-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.671154 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.671127    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ca9cc19-6578-4e8d-a424-61ed63f9b562-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.672931 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.672893    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.673019 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.672990    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.696321 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.696276    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jw5v\" (UniqueName: \"kubernetes.io/projected/9ca9cc19-6578-4e8d-a424-61ed63f9b562-kube-api-access-2jw5v\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:50.751763 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:50.751738    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:56:51.053404 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:51.053320    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"]
Oct 30 17:56:51.056768 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:56:51.056740    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca9cc19_6578_4e8d_a424_61ed63f9b562.slice/crio-de40dbceb56a5e9fe4b1878df19fef2f035cbf9585edaf3e0a1f48f21b061544 WatchSource:0}: Error finding container de40dbceb56a5e9fe4b1878df19fef2f035cbf9585edaf3e0a1f48f21b061544: Status 404 returned error can't find the container with id de40dbceb56a5e9fe4b1878df19fef2f035cbf9585edaf3e0a1f48f21b061544
Oct 30 17:56:51.230799 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:51.230762    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lvffz" event={"ID":"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f","Type":"ContainerStarted","Data":"55afabaddedd99c1b39c19210effbf92b4553cf14dcace98a064e70873dc4de7"}
Oct 30 17:56:51.231969 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:51.231939    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n" event={"ID":"9ca9cc19-6578-4e8d-a424-61ed63f9b562","Type":"ContainerStarted","Data":"de40dbceb56a5e9fe4b1878df19fef2f035cbf9585edaf3e0a1f48f21b061544"}
Oct 30 17:56:51.276250 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:51.276205    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lvffz" podStartSLOduration=1.567672073 podStartE2EDuration="4.276193695s" totalImagesPullingTime="2.708521622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 17:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:51.275322412 +0000 UTC m=+54.817933848" watchObservedRunningTime="2026-10-30 17:56:51.276193695 +0000 UTC m=+54.818805131"
Oct 30 17:56:51.382978 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:51.382955    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxg8q_04e386bc-2d2b-438e-bb10-af8b05ea5b30/dns-node-resolver/0.log"
Oct 30 17:56:52.172124 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:52.172098    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:56:52.583135 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:52.583058    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vxvtw_91a3053e-bb6f-4678-b865-4bb0e24f7dad/node-ca/0.log"
Oct 30 17:56:52.983484 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:52.983426    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lvffz_2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f/serve-healthcheck-canary/0.log"
Oct 30 17:56:53.238780 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:53.238697    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n" event={"ID":"9ca9cc19-6578-4e8d-a424-61ed63f9b562","Type":"ContainerStarted","Data":"40015e9c1cfc0662eca0e6ea841117aaabd5bce108ebd946ef31ebfe6ee111df"}
Oct 30 17:56:53.238780 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:53.238756    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n" event={"ID":"9ca9cc19-6578-4e8d-a424-61ed63f9b562","Type":"ContainerStarted","Data":"f0ace08a5bbe8e388305e1fd574751f92cb1e72020a6ec17688f1729755fc7be"}
Oct 30 17:56:53.285672 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:53.285626    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n" podStartSLOduration=2.145901677 podStartE2EDuration="3.285612802s" totalImagesPullingTime="1.139711125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 17:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:53.285355736 +0000 UTC m=+56.827967166" watchObservedRunningTime="2026-10-30 17:56:53.285612802 +0000 UTC m=+56.828224232"
Oct 30 17:56:55.134967 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.134897    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z7x2c"]
Oct 30 17:56:55.138146 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.138047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.205839 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.205806    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-metrics-client-ca\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.205993 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.205882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-root\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.205993 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.205967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-textfile\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.205993 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.205987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-tls\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.206092 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.206005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.206092 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.206024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-wtmp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.206154 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.206085    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-sys\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.206210 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.206194    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x5tp\" (UniqueName: \"kubernetes.io/projected/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-kube-api-access-5x5tp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.206247 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.206234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.207011 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.206997    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 17:56:55.207061 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.207004    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 17:56:55.207061 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.207002    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 17:56:55.207116 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.207060    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cclgx\""
Oct 30 17:56:55.306958 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.306925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-wtmp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.306958 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.306965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-sys\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307148 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.306984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x5tp\" (UniqueName: \"kubernetes.io/projected/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-kube-api-access-5x5tp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307148 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307148 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-metrics-client-ca\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307148 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307061    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-sys\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307148 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307105    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-wtmp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307305 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-root\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307305 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-textfile\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307305 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-tls\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307305 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307305 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307278    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-root\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307584 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307507    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-textfile\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307733 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307711    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-metrics-client-ca\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.307848 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.307831    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.309366 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.309349    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.309544 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.309516    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-tls\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.342121 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.342097    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x5tp\" (UniqueName: \"kubernetes.io/projected/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-kube-api-access-5x5tp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.452502 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:55.452424    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:56:55.463156 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:56:55.463113    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eeef7c4_96d5_4b07_8e8d_d8dff6385427.slice/crio-ec8c10345430e69c058f5ceb1cb0f2be8da23888104b6f38b89704686c2a5a1d WatchSource:0}: Error finding container ec8c10345430e69c058f5ceb1cb0f2be8da23888104b6f38b89704686c2a5a1d: Status 404 returned error can't find the container with id ec8c10345430e69c058f5ceb1cb0f2be8da23888104b6f38b89704686c2a5a1d
Oct 30 17:56:56.250345 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:56.250307    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7x2c" event={"ID":"2eeef7c4-96d5-4b07-8e8d-d8dff6385427","Type":"ContainerStarted","Data":"ec8c10345430e69c058f5ceb1cb0f2be8da23888104b6f38b89704686c2a5a1d"}
Oct 30 17:56:57.254094 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:57.254054    2576 generic.go:356] "Generic (PLEG): container finished" podID="2eeef7c4-96d5-4b07-8e8d-d8dff6385427" containerID="4f9fb162b9d50e20c155d81a82a737135fd2512061bcaebb26e5a5928c00bbe1" exitCode=0
Oct 30 17:56:57.254523 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:57.254098    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7x2c" event={"ID":"2eeef7c4-96d5-4b07-8e8d-d8dff6385427","Type":"ContainerDied","Data":"4f9fb162b9d50e20c155d81a82a737135fd2512061bcaebb26e5a5928c00bbe1"}
Oct 30 17:56:58.258890 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.258856    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7x2c" event={"ID":"2eeef7c4-96d5-4b07-8e8d-d8dff6385427","Type":"ContainerStarted","Data":"f68208d3bff1380189258aa25b96f57b5209fcb8bce4167ce5a04116ceb36e53"}
Oct 30 17:56:58.259316 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.258929    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7x2c" event={"ID":"2eeef7c4-96d5-4b07-8e8d-d8dff6385427","Type":"ContainerStarted","Data":"b218b27eaab0ca0123835d0a7a61ffab3207ec9521241642ceec09280d330c6b"}
Oct 30 17:56:58.336855 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.336805    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z7x2c" podStartSLOduration=2.4152141990000002 podStartE2EDuration="3.336791001s" totalImagesPullingTime="847.036412ms" totalInitContainerRuntime="74.54039ms" isStatefulPod=true podCreationTimestamp="2026-10-30 17:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:56:58.321879197 +0000 UTC m=+61.864490634" watchObservedRunningTime="2026-10-30 17:56:58.336791001 +0000 UTC m=+61.879402436"
Oct 30 17:56:58.671821 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.671786    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-64557c6d97-hgqx4"]
Oct 30 17:56:58.674842 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.674828    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.698095 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.698069    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 17:56:58.698095 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.698087    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 17:56:58.698276 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.698087    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 17:56:58.698276 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.698237    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 17:56:58.698380 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.698366    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 17:56:58.698446 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.698435    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dqs6z\""
Oct 30 17:56:58.727690 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.727659    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 17:56:58.727827 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.727752    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 17:56:58.728793 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.728771    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64557c6d97-hgqx4"]
Oct 30 17:56:58.737172 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.737153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.737228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.737178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.737228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.737195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.737228 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.737212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.737382 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.737366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.737418 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.737401    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.737503 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.737486    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.737585 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.737573    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.798670 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.798646    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 17:56:58.838193 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.838168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.838296 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.838197    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.838296 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.838240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.838296 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.838255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.838296 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.838274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.838296 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.838288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.839612 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.839186    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.839612 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.839330    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.839612 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.839523    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.839816 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.839677    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.839889 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.839769    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.840027 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.840006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.840209 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.840183    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.842098 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.842073    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.845451 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.845431    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.867275 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.867254    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:58.985710 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:58.985642    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:56:59.284584 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:56:59.284505    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64557c6d97-hgqx4"]
Oct 30 17:56:59.287367 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:56:59.287344    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c43e52_8525_45ad_9af8_026b029f538b.slice/crio-f7156936dbc3eef054d59472932ea7ad7caa7d8e53a270f84781cef90767b5c7 WatchSource:0}: Error finding container f7156936dbc3eef054d59472932ea7ad7caa7d8e53a270f84781cef90767b5c7: Status 404 returned error can't find the container with id f7156936dbc3eef054d59472932ea7ad7caa7d8e53a270f84781cef90767b5c7
Oct 30 17:57:00.235015 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.234985    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:00.266609 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.266572    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64557c6d97-hgqx4" event={"ID":"50c43e52-8525-45ad-9af8-026b029f538b","Type":"ContainerStarted","Data":"f7156936dbc3eef054d59472932ea7ad7caa7d8e53a270f84781cef90767b5c7"}
Oct 30 17:57:00.439033 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.439000    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"]
Oct 30 17:57:00.442313 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.442293    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.469988 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.469960    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 17:57:00.485431 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.485344    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 17:57:00.485431 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.485344    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 17:57:00.485431 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.485368    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 17:57:00.486122 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.486102    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-h85wj\""
Oct 30 17:57:00.486521 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.486501    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"]
Oct 30 17:57:00.499541 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.499510    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 17:57:00.499630 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.499606    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 17:57:00.554554 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.554452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ztqc\" (UniqueName: \"kubernetes.io/projected/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-kube-api-access-2ztqc\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.554554 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.554503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.554776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.554646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-metrics-client-ca\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.554776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.554689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.554776 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.554748    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.554957 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.554834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-federate-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.554957 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.554878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.555054 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.554955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-serving-certs-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.656218 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.656178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.656381 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.656247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-serving-certs-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.656381 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.656295    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ztqc\" (UniqueName: \"kubernetes.io/projected/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-kube-api-access-2ztqc\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.656381 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.656361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.656530 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.656411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-metrics-client-ca\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.656621 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.656600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.657122 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.656682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.657122 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.656727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-federate-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.657474 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.657451    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-serving-certs-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.657709 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.657684    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.657990 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.657969    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-metrics-client-ca\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.659341 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.659315    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.659662 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.659643    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.659725 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.659661    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-federate-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.660398 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.660375    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.683053 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.683032    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ztqc\" (UniqueName: \"kubernetes.io/projected/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-kube-api-access-2ztqc\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:00.759727 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:00.759661    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:01.100450 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:01.100417    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"]
Oct 30 17:57:01.117607 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:57:01.117577    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aac7377_2d25_4744_8fbf_49b60ba4d2c0.slice/crio-e69b758228cfce3f1ad8692f34b1e2661c96618a472b174269e9b95944430b64 WatchSource:0}: Error finding container e69b758228cfce3f1ad8692f34b1e2661c96618a472b174269e9b95944430b64: Status 404 returned error can't find the container with id e69b758228cfce3f1ad8692f34b1e2661c96618a472b174269e9b95944430b64
Oct 30 17:57:01.272434 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:01.272388    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7" event={"ID":"0aac7377-2d25-4744-8fbf-49b60ba4d2c0","Type":"ContainerStarted","Data":"e69b758228cfce3f1ad8692f34b1e2661c96618a472b174269e9b95944430b64"}
Oct 30 17:57:02.129491 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.129082    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:02.141582 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.141550    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.157793 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.157763    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:02.160241 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.160217    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 17:57:02.170895 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.170847    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 17:57:02.170895 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.170890    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 17:57:02.191023 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.190992    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 17:57:02.201007 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.200983    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 17:57:02.201150 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201063    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 17:57:02.201214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201183    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 17:57:02.201214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201202    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 17:57:02.201214 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201207    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 17:57:02.201353 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201187    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 17:57:02.201353 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201250    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 17:57:02.201353 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201286    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 17:57:02.201353 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201300    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 17:57:02.201353 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201289    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 17:57:02.201353 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201335    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 17:57:02.201574 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201391    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-69ncnlhdiul6u\""
Oct 30 17:57:02.201574 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.201444    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hqs45\""
Oct 30 17:57:02.271506 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271652 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271652 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271770 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271770 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271770 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271985 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271985 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271985 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.271985 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.271975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272191 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272191 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272029    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272191 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272191 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28mgj\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272191 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272191 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272470 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272470 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272594 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.272650 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.272616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373394 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373394 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373399    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373636 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373636 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373636 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373636 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373636 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373636 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373636 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373973 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28mgj\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373973 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373689    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373973 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373973 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373973 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373973 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373973 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.373973 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.374239 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.373978    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.374239 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.374005    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.374239 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.374035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.374524 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.374493    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.377961 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.377936    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.378079 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.377958    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.378424 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.378346    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.378514 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.378447    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.379146 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.379124    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.379948 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.379879    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.380501 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.380469    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.380887 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.380859    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.381562 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.381536    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.381927 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.381497    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.382120 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.382098    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.382264 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.382179    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.382674 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.382648    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.382869 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.382829    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.382869 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.382854    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.383059 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.382979    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.383654 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.383632    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.384470 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.384446    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.403548 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.403524    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28mgj\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.481257 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.481054    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:02.778944 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.778836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:57:02.779119 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.778931    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:02.795569 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.795541    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 17:57:02.795717 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.795663    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 17:57:02.802185 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.802159    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:02.802287 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.802168    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:57:02.839369 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.839327    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:02.856526 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:57:02.856495    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9e9544_bb16_400e_b614_f06894ad09da.slice/crio-222e997a856f88fd98b37103b71fd17597db420f212199c6b2f7a93ce9b1a56d WatchSource:0}: Error finding container 222e997a856f88fd98b37103b71fd17597db420f212199c6b2f7a93ce9b1a56d: Status 404 returned error can't find the container with id 222e997a856f88fd98b37103b71fd17597db420f212199c6b2f7a93ce9b1a56d
Oct 30 17:57:02.879734 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.879707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:57:02.896296 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.896256    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 17:57:02.916697 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.916652    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 17:57:02.923660 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:02.923638    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:57:03.048575 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.048553    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bkbt2\""
Oct 30 17:57:03.052707 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.052684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:57:03.055500 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.055479    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dxtgs\""
Oct 30 17:57:03.059713 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.059692    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:03.061003 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.060982    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-tzlmk\""
Oct 30 17:57:03.064847 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.064829    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:57:03.283416 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.283340    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerStarted","Data":"222e997a856f88fd98b37103b71fd17597db420f212199c6b2f7a93ce9b1a56d"}
Oct 30 17:57:03.285360 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.285334    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7" event={"ID":"0aac7377-2d25-4744-8fbf-49b60ba4d2c0","Type":"ContainerStarted","Data":"97cfa6bd298796e5c650404265407fa56a9f8aac74d3749d3bece2708696c7cc"}
Oct 30 17:57:03.287678 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.287655    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64557c6d97-hgqx4" event={"ID":"50c43e52-8525-45ad-9af8-026b029f538b","Type":"ContainerStarted","Data":"d33c43a3bc463ca30bffcc883b8080288678359d574f7f30a60b3a5b229743bb"}
Oct 30 17:57:03.340063 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.340018    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-64557c6d97-hgqx4" podStartSLOduration=2.399225568 podStartE2EDuration="5.340004033s" totalImagesPullingTime="2.940778465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 17:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:57:03.339226348 +0000 UTC m=+66.881837783" watchObservedRunningTime="2026-10-30 17:57:03.340004033 +0000 UTC m=+66.882615473"
Oct 30 17:57:03.363920 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.363872    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dttj5"]
Oct 30 17:57:03.366960 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:57:03.366928    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3c695e_3fac_49f4_838e_251b0c17fe67.slice/crio-44839f079e230e73e1588d9c1be0c51a6134dbb59817cd8dd5b3c55cadef8281 WatchSource:0}: Error finding container 44839f079e230e73e1588d9c1be0c51a6134dbb59817cd8dd5b3c55cadef8281: Status 404 returned error can't find the container with id 44839f079e230e73e1588d9c1be0c51a6134dbb59817cd8dd5b3c55cadef8281
Oct 30 17:57:03.392176 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.392151    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d9ml4"]
Oct 30 17:57:03.393010 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.392983    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-trnpq"]
Oct 30 17:57:03.397993 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:57:03.397956    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba85cd81_0f4d_4907_9190_8ba2e3199aa3.slice/crio-a7e4861b61c44604c8b00f069806ea3ca8b46c59d3493196796b5df884133b8c WatchSource:0}: Error finding container a7e4861b61c44604c8b00f069806ea3ca8b46c59d3493196796b5df884133b8c: Status 404 returned error can't find the container with id a7e4861b61c44604c8b00f069806ea3ca8b46c59d3493196796b5df884133b8c
Oct 30 17:57:03.400210 ip-10-0-131-31 kubenswrapper[2576]: W1030 17:57:03.400192    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08444539_c0d4_4b26_9bbd_c75c21b32a21.slice/crio-be8a97409d9e8d089dc721f01cb1cb97a8c2aaf56384b43057091f605a8a0525 WatchSource:0}: Error finding container be8a97409d9e8d089dc721f01cb1cb97a8c2aaf56384b43057091f605a8a0525: Status 404 returned error can't find the container with id be8a97409d9e8d089dc721f01cb1cb97a8c2aaf56384b43057091f605a8a0525
Oct 30 17:57:03.688395 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:03.688360    2576 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64557c6d97-hgqx4"]
Oct 30 17:57:04.295556 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:04.295519    2576 generic.go:356] "Generic (PLEG): container finished" podID="7c9e9544-bb16-400e-b614-f06894ad09da" containerID="b75756a6b05f4a5b01290d9a5a86376e4d82c40388d9b932e64cf8029d3fd207" exitCode=0
Oct 30 17:57:04.296037 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:04.295642    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerDied","Data":"b75756a6b05f4a5b01290d9a5a86376e4d82c40388d9b932e64cf8029d3fd207"}
Oct 30 17:57:04.299593 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:04.299562    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7" event={"ID":"0aac7377-2d25-4744-8fbf-49b60ba4d2c0","Type":"ContainerStarted","Data":"b8fce2ee13a607f3b0841328af3d5a1a8f37e17ac1a5d710fcd6210d3b527954"}
Oct 30 17:57:04.299675 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:04.299625    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7" event={"ID":"0aac7377-2d25-4744-8fbf-49b60ba4d2c0","Type":"ContainerStarted","Data":"e405cf3e5d968adaa7de139f8a4a89cedbbedc1d514429135cdd6689b516d803"}
Oct 30 17:57:04.301960 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:04.301935    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d9ml4" event={"ID":"ba85cd81-0f4d-4907-9190-8ba2e3199aa3","Type":"ContainerStarted","Data":"a7e4861b61c44604c8b00f069806ea3ca8b46c59d3493196796b5df884133b8c"}
Oct 30 17:57:04.303386 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:04.303355    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-trnpq" event={"ID":"08444539-c0d4-4b26-9bbd-c75c21b32a21","Type":"ContainerStarted","Data":"be8a97409d9e8d089dc721f01cb1cb97a8c2aaf56384b43057091f605a8a0525"}
Oct 30 17:57:04.304975 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:04.304948    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dttj5" event={"ID":"1e3c695e-3fac-49f4-838e-251b0c17fe67","Type":"ContainerStarted","Data":"44839f079e230e73e1588d9c1be0c51a6134dbb59817cd8dd5b3c55cadef8281"}
Oct 30 17:57:04.437275 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:04.437221    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7" podStartSLOduration=1.582536598 podStartE2EDuration="4.437204535s" totalImagesPullingTime="2.854667937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 17:57:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:57:04.436998427 +0000 UTC m=+67.979609890" watchObservedRunningTime="2026-10-30 17:57:04.437204535 +0000 UTC m=+67.979815970"
Oct 30 17:57:05.312571 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:05.312527    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dttj5" event={"ID":"1e3c695e-3fac-49f4-838e-251b0c17fe67","Type":"ContainerStarted","Data":"e187bb9726c2000ca592c920f5267d90392abb6703aa893918927dad5fc2f1b8"}
Oct 30 17:57:05.312571 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:05.312576    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dttj5" event={"ID":"1e3c695e-3fac-49f4-838e-251b0c17fe67","Type":"ContainerStarted","Data":"e932590c09e4f517f896cbf3043b28ce8e45dbfba25b19ec44661110fc3f3cda"}
Oct 30 17:57:05.359348 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:05.359259    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dttj5" podStartSLOduration=67.203474149 podStartE2EDuration="1m8.359238621s" totalImagesPullingTime="1.155764472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:57:05.358012998 +0000 UTC m=+68.900624435" watchObservedRunningTime="2026-10-30 17:57:05.359238621 +0000 UTC m=+68.901850049"
Oct 30 17:57:08.327579 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:08.327479    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d9ml4" event={"ID":"ba85cd81-0f4d-4907-9190-8ba2e3199aa3","Type":"ContainerStarted","Data":"d88eac1b26e7d4de5e7080bb4cf058e025a0f805abd9e8ca230c949a02d2f217"}
Oct 30 17:57:08.327579 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:08.327547    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:57:08.329800 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:08.329773    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerStarted","Data":"c63380af2646ba58fc4a6ba1c0edccd9dc5cabb3c2dda5e4496daa4d09bac399"}
Oct 30 17:57:08.373518 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:08.373467    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d9ml4" podStartSLOduration=67.298764509 podStartE2EDuration="1m11.373452268s" totalImagesPullingTime="4.074687759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 17:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:57:08.372972082 +0000 UTC m=+71.915583521" watchObservedRunningTime="2026-10-30 17:57:08.373452268 +0000 UTC m=+71.916063704"
Oct 30 17:57:08.986009 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:08.985969    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:09.337054 ip-10-0-131-31 kubenswrapper[2576]: I1030 17:57:09.336882    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerStarted","Data":"aa5f3e44c20d35aebaf7c71c801b093fe3fd4e880b818d766986f66acfb19ecb"}
Oct 30 17:57:09.757829 ip-10-0-131-31 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 17:57:09.775779 ip-10-0-131-31 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 17:57:09.776051 ip-10-0-131-31 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 17:57:09.776115 ip-10-0-131-31 systemd[1]: kubelet.service: Consumed 3.880s CPU time.
Oct 30 17:57:09.784645 ip-10-0-131-31 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 17:57:09.869573 ip-10-0-131-31 kubenswrapper[5746]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 17:57:09.869573 ip-10-0-131-31 kubenswrapper[5746]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 17:57:09.869573 ip-10-0-131-31 kubenswrapper[5746]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 17:57:09.869573 ip-10-0-131-31 kubenswrapper[5746]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 17:57:09.873058 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.872964    5746 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:57:09.873058 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873055    5746 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:57:09.873058 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873059    5746 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:57:09.873058 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873063    5746 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873066    5746 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873069    5746 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873072    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873074    5746 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873077    5746 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873080    5746 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873083    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873086    5746 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873088    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873144    5746 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873147    5746 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873152    5746 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873155    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873158    5746 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873161    5746 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873164    5746 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873166    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873169    5746 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:57:09.873276 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873173    5746 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873176    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873179    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873181    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873187    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873190    5746 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873193    5746 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873196    5746 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873198    5746 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873201    5746 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873204    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873206    5746 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873209    5746 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873211    5746 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873214    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873217    5746 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873222    5746 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873224    5746 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873227    5746 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873230    5746 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:57:09.873994 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873233    5746 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873235    5746 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873238    5746 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873241    5746 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873244    5746 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873247    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873250    5746 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873253    5746 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873256    5746 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873261    5746 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873264    5746 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873266    5746 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873269    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873272    5746 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873274    5746 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873278    5746 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873280    5746 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873283    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873286    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873289    5746 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:57:09.874490 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873291    5746 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873294    5746 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873299    5746 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873302    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873304    5746 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873307    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873309    5746 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873312    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873315    5746 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873317    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873320    5746 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873323    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873326    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873328    5746 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873336    5746 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873340    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873343    5746 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873345    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873348    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:57:09.874984 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873353    5746 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873356    5746 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873359    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873362    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873365    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873368    5746 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873371    5746 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873376    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873379    5746 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873382    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873386    5746 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873391    5746 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873395    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873399    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873403    5746 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873406    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873409    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873413    5746 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873419    5746 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:57:09.875457 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873422    5746 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873424    5746 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873427    5746 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873430    5746 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873432    5746 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.873435    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875745    5746 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875756    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875759    5746 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875762    5746 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875764    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875767    5746 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875770    5746 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875773    5746 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875797    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875805    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875810    5746 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875816    5746 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875821    5746 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:57:09.876168 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875832    5746 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875838    5746 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875843    5746 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875846    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875851    5746 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875856    5746 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875861    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875866    5746 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875871    5746 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875876    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875881    5746 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875885    5746 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875892    5746 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875895    5746 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875898    5746 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875901    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875922    5746 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875927    5746 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875933    5746 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875938    5746 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:57:09.876707 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875941    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875944    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875947    5746 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875950    5746 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875953    5746 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875959    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875961    5746 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875964    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875968    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875971    5746 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875974    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875976    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875979    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875982    5746 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875984    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875987    5746 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875990    5746 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875992    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.875997    5746 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876000    5746 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:57:09.877257 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876002    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876005    5746 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876008    5746 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876010    5746 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876013    5746 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876016    5746 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876019    5746 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876022    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876025    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876030    5746 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876037    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876040    5746 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876043    5746 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876046    5746 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876049    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876052    5746 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876118    5746 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876122    5746 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:57:09.877832 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876126    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876129    5746 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876132    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876139    5746 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876142    5746 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876145    5746 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876147    5746 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876150    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876153    5746 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876155    5746 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876158    5746 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876163    5746 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876166    5746 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876169    5746 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876171    5746 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876174    5746 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876177    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876180    5746 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876183    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876186    5746 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:57:09.878338 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876188    5746 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876191    5746 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876194    5746 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876199    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876202    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876205    5746 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876207    5746 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876210    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876214    5746 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876217    5746 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876219    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876222    5746 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876227    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876230    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.876232    5746 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878371    5746 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878390    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878396    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878401    5746 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878407    5746 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:57:09.878868 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878412    5746 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878418    5746 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878423    5746 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878429    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878434    5746 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878440    5746 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878445    5746 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878450    5746 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878455    5746 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878465    5746 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878470    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878474    5746 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878479    5746 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878484    5746 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878490    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878495    5746 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878500    5746 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878505    5746 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878510    5746 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:57:09.879542 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878518    5746 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878525    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878536    5746 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878542    5746 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878549    5746 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878555    5746 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878560    5746 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878566    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878570    5746 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878577    5746 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878582    5746 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878588    5746 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878593    5746 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878598    5746 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878609    5746 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878613    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878618    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878623    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878628    5746 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:57:09.880055 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878633    5746 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878638    5746 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878643    5746 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878648    5746 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878653    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878658    5746 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878662    5746 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878667    5746 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878763    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878768    5746 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878773    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878778    5746 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878783    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878788    5746 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878793    5746 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878798    5746 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878802    5746 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878807    5746 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878812    5746 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878818    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:57:09.880548 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878822    5746 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878832    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878837    5746 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878842    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878848    5746 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878853    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878859    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878864    5746 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878869    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878874    5746 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878879    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878884    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878889    5746 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878894    5746 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878922    5746 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878930    5746 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878936    5746 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878941    5746 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878946    5746 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878951    5746 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:57:09.881121 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878956    5746 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878961    5746 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878966    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878971    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878976    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878985    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878990    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.878995    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879000    5746 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879005    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879010    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879015    5746 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879020    5746 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879025    5746 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879030    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879035    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879041    5746 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879048    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879060    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:57:09.881970 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879065    5746 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879070    5746 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879075    5746 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.879079    5746 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880654    5746 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880674    5746 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880681    5746 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880686    5746 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880691    5746 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880694    5746 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880699    5746 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880705    5746 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880708    5746 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880713    5746 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880716    5746 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880721    5746 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880726    5746 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880729    5746 flags.go:64] FLAG: --cgroup-root=""
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880734    5746 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880737    5746 flags.go:64] FLAG: --client-ca-file=""
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880741    5746 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880744    5746 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880750    5746 flags.go:64] FLAG: --cluster-domain=""
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880753    5746 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 17:57:09.882690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880756    5746 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880759    5746 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880763    5746 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880768    5746 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880771    5746 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880777    5746 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880781    5746 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880784    5746 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880787    5746 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880790    5746 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880793    5746 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880796    5746 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880801    5746 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880804    5746 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880808    5746 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880810    5746 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880813    5746 flags.go:64] FLAG: --enable-server="true"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880816    5746 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880821    5746 flags.go:64] FLAG: --event-burst="100"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880824    5746 flags.go:64] FLAG: --event-qps="50"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880826    5746 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880829    5746 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880832    5746 flags.go:64] FLAG: --eviction-hard=""
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880836    5746 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 17:57:09.883795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880839    5746 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880842    5746 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880845    5746 flags.go:64] FLAG: --eviction-soft=""
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880849    5746 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880851    5746 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880854    5746 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880857    5746 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880860    5746 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880863    5746 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880866    5746 flags.go:64] FLAG: --feature-gates=""
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880870    5746 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880873    5746 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880877    5746 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880880    5746 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880883    5746 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880886    5746 flags.go:64] FLAG: --help="false"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880889    5746 flags.go:64] FLAG: --hostname-override="ip-10-0-131-31.ec2.internal"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880892    5746 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880895    5746 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880898    5746 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880927    5746 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880932    5746 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880935    5746 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 17:57:09.885124 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880938    5746 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880941    5746 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880944    5746 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880947    5746 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880950    5746 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880953    5746 flags.go:64] FLAG: --kube-reserved=""
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880956    5746 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880958    5746 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880961    5746 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880965    5746 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880967    5746 flags.go:64] FLAG: --lock-file=""
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880970    5746 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880973    5746 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880977    5746 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880988    5746 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880991    5746 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880994    5746 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.880997    5746 flags.go:64] FLAG: --logging-format="text"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881000    5746 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881004    5746 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881007    5746 flags.go:64] FLAG: --manifest-url=""
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881009    5746 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881014    5746 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881017    5746 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881021    5746 flags.go:64] FLAG: --max-pods="110"
Oct 30 17:57:09.886163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881023    5746 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881026    5746 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881031    5746 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881035    5746 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881037    5746 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881040    5746 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881043    5746 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881051    5746 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881055    5746 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881057    5746 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881060    5746 flags.go:64] FLAG: --pod-cidr=""
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881063    5746 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881070    5746 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881073    5746 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881076    5746 flags.go:64] FLAG: --port="10250"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881080    5746 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881083    5746 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02f3a9b1604f76372"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881086    5746 flags.go:64] FLAG: --qos-reserved=""
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881089    5746 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881092    5746 flags.go:64] FLAG: --register-node="true"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881095    5746 flags.go:64] FLAG: --register-with-taints=""
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881099    5746 flags.go:64] FLAG: --registry-burst="10"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881102    5746 flags.go:64] FLAG: --registry-qps="5"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881104    5746 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881107    5746 flags.go:64] FLAG: --reserved-memory=""
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881111    5746 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 17:57:09.887287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881114    5746 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881117    5746 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881120    5746 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881123    5746 flags.go:64] FLAG: --runonce="false"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881126    5746 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881129    5746 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881132    5746 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881135    5746 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881138    5746 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881142    5746 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881145    5746 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881148    5746 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881151    5746 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881154    5746 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881156    5746 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881159    5746 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881162    5746 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881165    5746 flags.go:64] FLAG: --system-cgroups=""
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881169    5746 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881175    5746 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881178    5746 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881183    5746 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.881190    5746 flags.go:64] FLAG: --tls-min-version=""
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882055    5746 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882069    5746 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 17:57:09.888587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882075    5746 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882082    5746 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882088    5746 flags.go:64] FLAG: --v="2"
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882095    5746 flags.go:64] FLAG: --version="false"
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882099    5746 flags.go:64] FLAG: --vmodule=""
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882106    5746 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882113    5746 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882279    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882284    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882287    5746 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882290    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882292    5746 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882295    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882297    5746 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882300    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882303    5746 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882305    5746 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882310    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882313    5746 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882316    5746 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882318    5746 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:57:09.889771 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882321    5746 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882323    5746 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882326    5746 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882329    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882331    5746 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882334    5746 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882336    5746 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882339    5746 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882342    5746 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882345    5746 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882348    5746 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882350    5746 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882353    5746 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882356    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882358    5746 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882361    5746 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882364    5746 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882366    5746 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882369    5746 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882372    5746 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:57:09.890745 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882374    5746 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882377    5746 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882380    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882382    5746 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882384    5746 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882387    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882389    5746 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882392    5746 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882394    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882397    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882400    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882403    5746 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882405    5746 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882408    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882410    5746 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882413    5746 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882415    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882418    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882420    5746 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:57:09.891671 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882423    5746 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882426    5746 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882428    5746 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882431    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882436    5746 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882439    5746 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882441    5746 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882444    5746 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882447    5746 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882450    5746 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882453    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882457    5746 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882461    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882465    5746 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882469    5746 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882474    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882476    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882479    5746 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882483    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882487    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:57:09.892535 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882491    5746 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882495    5746 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882499    5746 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882505    5746 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882509    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882513    5746 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882517    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882521    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882526    5746 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882530    5746 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882534    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882539    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882543    5746 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882548    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882552    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882557    5746 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882563    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882568    5746 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882572    5746 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:57:09.893451 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882577    5746 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882581    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882586    5746 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882590    5746 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882594    5746 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882599    5746 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882603    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882607    5746 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882611    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882618    5746 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882626    5746 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882633    5746 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882638    5746 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882643    5746 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.882689    5746 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 17:57:09.894449 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882755    5746 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882762    5746 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882770    5746 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882775    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882779    5746 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882784    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882788    5746 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882792    5746 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882796    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882800    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882805    5746 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882809    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882813    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882817    5746 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882821    5746 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882827    5746 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882831    5746 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882836    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882840    5746 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882843    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:57:09.895993 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882848    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882852    5746 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882856    5746 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882860    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882864    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882868    5746 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882872    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882876    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882880    5746 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882884    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882888    5746 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882893    5746 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882897    5746 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882901    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882929    5746 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882934    5746 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882939    5746 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882944    5746 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882948    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:57:09.896465 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882952    5746 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882956    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882964    5746 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882973    5746 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882980    5746 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882987    5746 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882994    5746 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.882999    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883006    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883011    5746 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883016    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883020    5746 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883025    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883030    5746 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883034    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883038    5746 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883042    5746 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883047    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:57:09.897123 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883052    5746 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883057    5746 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883062    5746 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883066    5746 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883070    5746 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883075    5746 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883079    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883083    5746 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883088    5746 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883092    5746 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883096    5746 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883102    5746 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883107    5746 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883112    5746 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883116    5746 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883121    5746 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883133    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883141    5746 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883145    5746 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883149    5746 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:57:09.897894 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883154    5746 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883159    5746 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883166    5746 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883171    5746 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883175    5746 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883180    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883185    5746 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883189    5746 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883194    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883198    5746 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883203    5746 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883207    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883212    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883217    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883223    5746 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883227    5746 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883233    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883237    5746 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883242    5746 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:57:09.898754 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883246    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883250    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883255    5746 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883261    5746 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883266    5746 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883271    5746 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883276    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883281    5746 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883286    5746 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.883291    5746 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.899151    5746 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.899169    5746 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.899194    5746 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.899203    5746 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899288    5746 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899298    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899304    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899311    5746 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899317    5746 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899322    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:57:09.899589 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899326    5746 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899330    5746 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899335    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899339    5746 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899344    5746 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899348    5746 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899352    5746 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899357    5746 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899361    5746 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899365    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899369    5746 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899374    5746 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899378    5746 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899382    5746 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899387    5746 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899391    5746 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899396    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899400    5746 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899404    5746 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899409    5746 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:57:09.900513 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899414    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899418    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899423    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899429    5746 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899434    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899438    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899442    5746 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899446    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899451    5746 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899455    5746 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899459    5746 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899463    5746 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899467    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899472    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899476    5746 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899481    5746 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899485    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899489    5746 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899493    5746 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899497    5746 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:57:09.901473 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899502    5746 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899506    5746 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899510    5746 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899515    5746 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899519    5746 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899524    5746 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899528    5746 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899533    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899537    5746 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899541    5746 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899545    5746 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899549    5746 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899556    5746 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899563    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899569    5746 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899573    5746 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899579    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899583    5746 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899587    5746 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899591    5746 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:57:09.902329 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899595    5746 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899599    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899603    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899607    5746 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899611    5746 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899616    5746 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899620    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899624    5746 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899629    5746 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899633    5746 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899637    5746 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899641    5746 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899645    5746 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899650    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899654    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899659    5746 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899663    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899667    5746 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899672    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:57:09.903190 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899676    5746 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899680    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899684    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899689    5746 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899693    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899697    5746 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899701    5746 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899706    5746 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899710    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899714    5746 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899720    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899724    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899728    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899732    5746 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899736    5746 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899740    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899744    5746 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899749    5746 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899753    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:57:09.904028 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899757    5746 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.899761    5746 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.899807    5746 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900150    5746 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900162    5746 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900167    5746 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900172    5746 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900176    5746 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900180    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900184    5746 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900188    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900192    5746 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900196    5746 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900201    5746 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900205    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:57:09.904587 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900212    5746 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900219    5746 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900223    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900228    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900234    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900238    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900242    5746 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900246    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900250    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900256    5746 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900261    5746 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900265    5746 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900269    5746 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900273    5746 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900277    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900281    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900285    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900289    5746 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900294    5746 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900298    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:57:09.905059 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900302    5746 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900306    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900310    5746 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900314    5746 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900318    5746 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900322    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900326    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900333    5746 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900338    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900342    5746 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900346    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900350    5746 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900354    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900358    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900363    5746 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900367    5746 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900371    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900375    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900380    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:57:09.905877 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900383    5746 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900388    5746 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900392    5746 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900398    5746 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900402    5746 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900407    5746 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900411    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900415    5746 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900419    5746 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900423    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900427    5746 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900431    5746 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900436    5746 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900440    5746 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900444    5746 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900448    5746 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900452    5746 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900456    5746 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900461    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:57:09.906656 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900465    5746 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900469    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900473    5746 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900477    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900481    5746 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900485    5746 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900490    5746 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900494    5746 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900498    5746 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900502    5746 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900506    5746 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900510    5746 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900514    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900518    5746 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900522    5746 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900526    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900530    5746 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900539    5746 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900544    5746 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900548    5746 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:57:09.907426 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900552    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900557    5746 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900561    5746 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900565    5746 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900569    5746 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900573    5746 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900577    5746 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900581    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900585    5746 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900589    5746 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900593    5746 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900598    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900602    5746 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900606    5746 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900611    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900615    5746 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:57:09.908064 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.900657    5746 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900741    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900748    5746 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900753    5746 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900757    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900761    5746 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900766    5746 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900770    5746 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900774    5746 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900778    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900783    5746 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900787    5746 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900791    5746 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900796    5746 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900801    5746 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900805    5746 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900812    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900816    5746 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900821    5746 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900825    5746 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 17:57:09.908534 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900829    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900833    5746 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900838    5746 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900842    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900846    5746 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900851    5746 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900855    5746 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900859    5746 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900863    5746 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900867    5746 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900871    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900876    5746 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900880    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900884    5746 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900888    5746 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900893    5746 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900897    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900922    5746 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900929    5746 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 17:57:09.909398 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900933    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900937    5746 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900941    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900945    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900949    5746 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900953    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900957    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900961    5746 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900966    5746 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900973    5746 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900978    5746 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900982    5746 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900986    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900991    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.900995    5746 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901000    5746 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901004    5746 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901008    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901012    5746 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901016    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 17:57:09.910234 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901021    5746 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901025    5746 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901029    5746 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901033    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901038    5746 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901046    5746 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901052    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901057    5746 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901062    5746 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901066    5746 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901070    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901074    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901079    5746 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901083    5746 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901088    5746 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901092    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901097    5746 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901101    5746 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901106    5746 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901110    5746 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 17:57:09.910805 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901114    5746 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901121    5746 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901125    5746 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901130    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901134    5746 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901138    5746 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901142    5746 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901147    5746 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901151    5746 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901155    5746 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901160    5746 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901164    5746 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901169    5746 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901173    5746 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901177    5746 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901181    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901185    5746 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901190    5746 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901194    5746 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 17:57:09.911524 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901198    5746 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901202    5746 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901207    5746 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901211    5746 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901217    5746 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901221    5746 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901225    5746 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901229    5746 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:09.901234    5746 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.901314    5746 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.902958    5746 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.903030    5746 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.903297    5746 server.go:1014] "Starting client certificate rotation"
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.903461    5746 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.903519    5746 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 17:50:57 +0000 UTC" deadline="2028-04-02 11:27:22.722343933 +0000 UTC"
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.903562    5746 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12473h30m12.818784465s"
Oct 30 17:57:09.912071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.904034    5746 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 17:57:09.912537 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.906542    5746 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 17:57:09.912537 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.908001    5746 log.go:25] "Validated CRI v1 runtime API"
Oct 30 17:57:09.912537 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.910356    5746 log.go:25] "Validated CRI v1 image API"
Oct 30 17:57:09.912537 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.911191    5746 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 17:57:09.913362 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.913339    5746 fs.go:135] Filesystem UUIDs: map[475bd308-c0c5-4f3a-9f15-044b048b51f7:/dev/nvme0n1p4 49d15a24-7cfa-4a97-abbc-3dc9814f3948:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 30 17:57:09.913714 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.913363    5746 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/04353ae0bb37b5944719e4306001e51e44a5b7fa7619139a861eb0146f6b33ab/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/04353ae0bb37b5944719e4306001e51e44a5b7fa7619139a861eb0146f6b33ab/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/222e997a856f88fd98b37103b71fd17597db420f212199c6b2f7a93ce9b1a56d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/222e997a856f88fd98b37103b71fd17597db420f212199c6b2f7a93ce9b1a56d/userdata/shm major:0 minor:319 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3282e78544ee7ab35e0a2d9aaeb0daf14e087e263bdccee24698a52137f4d6ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3282e78544ee7ab35e0a2d9aaeb0daf14e087e263bdccee24698a52137f4d6ae/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/44839f079e230e73e1588d9c1be0c51a6134dbb59817cd8dd5b3c55cadef8281/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/44839f079e230e73e1588d9c1be0c51a6134dbb59817cd8dd5b3c55cadef8281/userdata/shm major:0 minor:328 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/55b146394f9275007f459e946a522f4fe86e682c048c213a905141fb99fddc46/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/55b146394f9275007f459e946a522f4fe86e682c048c213a905141fb99fddc46/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/61284ced422bbbce93df64f4c4a520ec2d52427cc95ee8ec7885a6993a7ebd53/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/61284ced422bbbce93df64f4c4a520ec2d52427cc95ee8ec7885a6993a7ebd53/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74f8ab7cef4403c5e87a4457a33a5b13030373364ae5a1b94b0ecb34a6755af8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74f8ab7cef4403c5e87a4457a33a5b13030373364ae5a1b94b0ecb34a6755af8/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/762289fdc4e13abdfc08e25ba85382d613aa87b23eccdcd31cd44b7e51aebc9d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/762289fdc4e13abdfc08e25ba85382d613aa87b23eccdcd31cd44b7e51aebc9d/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/892bff89c7902ee56649cc1734215455296c65b3256ddd266f83d468c6d1ce63/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/892bff89c7902ee56649cc1734215455296c65b3256ddd266f83d468c6d1ce63/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/90dddc2bf58d76d9a42310ecc46a91a129e4e43c7f28020fe20c3c12e311a499/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/90dddc2bf58d76d9a42310ecc46a91a129e4e43c7f28020fe20c3c12e311a499/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/94750cd6b25ccb4311f07f812adc895276b1fa831e3df0a40f14703974234cd3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/94750cd6b25ccb4311f07f812adc895276b1fa831e3df0a40f14703974234cd3/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a45417c32b7d6f944c419a810f2ee280f977b8c5d434aaf220b4e76eaea45b6c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a45417c32b7d6f944c419a810f2ee280f977b8c5d434aaf220b4e76eaea45b6c/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a7e4861b61c44604c8b00f069806ea3ca8b46c59d3493196796b5df884133b8c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a7e4861b61c44604c8b00f069806ea3ca8b46c59d3493196796b5df884133b8c/userdata/shm major:0 minor:335 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/be8a97409d9e8d089dc721f01cb1cb97a8c2aaf56384b43057091f605a8a0525/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/be8a97409d9e8d089dc721f01cb1cb97a8c2aaf56384b43057091f605a8a0525/userdata/shm major:0 minor:334 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de40dbceb56a5e9fe4b1878df19fef2f035cbf9585edaf3e0a1f48f21b061544/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de40dbceb56a5e9fe4b1878df19fef2f035cbf9585edaf3e0a1f48f21b061544/userdata/shm major:0 minor:254 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e69b758228cfce3f1ad8692f34b1e2661c96618a472b174269e9b95944430b64/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e69b758228cfce3f1ad8692f34b1e2661c96618a472b174269e9b95944430b64/userdata/shm major:0 minor:297 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ec8c10345430e69c058f5ceb1cb0f2be8da23888104b6f38b89704686c2a5a1d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ec8c10345430e69c058f5ceb1cb0f2be8da23888104b6f38b89704686c2a5a1d/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5023e818d109e36e9626caf775d474710489b4424ef9a21e15aaf51aedef629/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5023e818d109e36e9626caf775d474710489b4424ef9a21e15aaf51aedef629/userdata/shm major:0 minor:100 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f7156936dbc3eef054d59472932ea7ad7caa7d8e53a270f84781cef90767b5c7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f7156936dbc3eef054d59472932ea7ad7caa7d8e53a270f84781cef90767b5c7/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f9f5382ca9626b9391fd289f9b925080bd3059cb8255c9a462e0bdc1aacefb3e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f9f5382ca9626b9391fd289f9b925080bd3059cb8255c9a462e0bdc1aacefb3e/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fca71c4819366726fe0e60c275913c31b866a2c8e051c60a82d9cad082d8faf4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fca71c4819366726fe0e60c275913c31b866a2c8e051c60a82d9cad082d8faf4/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/04e386bc-2d2b-438e-bb10-af8b05ea5b30/volumes/kubernetes.io~projected/kube-api-access-8hq7z:{mountpoint:/var/lib/kubelet/pods/04e386bc-2d2b-438e-bb10-af8b05ea5b30/volumes/kubernetes.io~projected/kube-api-access-8hq7z major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08444539-c0d4-4b26-9bbd-c75c21b32a21/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/08444539-c0d4-4b26-9bbd-c75c21b32a21/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~projected/kube-api-access-2ztqc:{mountpoint:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~projected/kube-api-access-2ztqc major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volumes/kubernetes.io~projected/kube-api-access-vc6lk:{mountpoint:/var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volumes/kubernetes.io~projected/kube-api-access-vc6lk major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b/volumes/kubernetes.io~projected/kube-api-access-fm9zl:{mountpoint:/var/lib/kubelet/pods/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b/volumes/kubernetes.io~projected/kube-api-access-fm9zl major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e3c695e-3fac-49f4-838e-251b0c17fe67/volumes/kubernetes.io~projected/kube-api-access-vwf77:{mountpoint:/var/lib/kubelet/pods/1e3c695e-3fac-49f4-838e-251b0c17fe67/volumes/kubernetes.io~projected/kube-api-access-vwf77 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e3c695e-3fac-49f4-838e-251b0c17fe67/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/1e3c695e-3fac-49f4-838e-251b0c17fe67/volumes/kubernetes.io~secret/metrics-certs major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab/volumes/kubernetes.io~projected/kube-api-access-r9j2g:{mountpoint:/var/lib/kubelet/pods/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab/volumes/kubernetes.io~projected/kube-api-access-r9j2g major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f/volumes/kubernetes.io~projected/kube-api-access-fdd5b:{mountpoint:/var/lib/kubelet/pods/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f/volumes/kubernetes.io~projected/kube-api-access-fdd5b major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f/volumes/kubernetes.io~secret/cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~projected/kube-api-access-5x5tp:{mountpoint:/var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~projected/kube-api-access-5x5tp major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~projected/kube-api-access-2z8q6:{mountpoint:/var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~projected/kube-api-access-2z8q6 major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~projected/kube-api-access-p5424:{mountpoint:/var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~projected/kube-api-access-p5424 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/config-reloader/2 major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/prometheus/15 major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~empty-dir/config-out major:0 minor:308 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~projected/kube-api-access-28mgj:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~projected/kube-api-access-28mgj major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~projected/tls-assets major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/config major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:316 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:309 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:311 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:317 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/web-config major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91a3053e-bb6f-4678-b865-4bb0e24f7dad/volumes/kubernetes.io~projected/kube-api-access-th8x7:{mountpoint:/var/lib/kubelet/pods/91a3053e-bb6f-4678-b865-4bb0e24f7dad/volumes/kubernetes.io~projected/kube-api-access-th8x7 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~projected/kube-api-access-2jw5v:{mountpoint:/var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~projected/kube-api-access-2jw5v major:0 minor:246 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:245 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a304069c-7b5f-4ddf-9549-33797ac13658/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/a304069c-7b5f-4ddf-9549-33797ac13658/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6ee33fb-66e9-4483-b3c1-a289e9620de4/volumes/kubernetes.io~projected/kube-api-access-x42gn:{mountpoint:/var/lib/kubelet/pods/a6ee33fb-66e9-4483-b3c1-a289e9620de4/volumes/kubernetes.io~projected/kube-api-access-x42gn major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6ee33fb-66e9-4483-b3c1-a289e9620de4/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/a6ee33fb-66e9-4483-b3c1-a289e9620de4/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ba85cd81-0f4d-4907-9190-8ba2e3199aa3/volumes/kubernetes.io~projected/kube-api-access-nqhkd:{mountpoint:/var/lib/kubelet/pods/ba85cd81-0f4d-4907-9190-8ba2e3199aa3/volumes/kubernetes.io~projected/kube-api-access-nqhkd major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3/volumes/kubernetes.io~projected/kube-api-access-khb5q:{mountpoint:/var/lib/kubelet/pods/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3/volumes/kubernetes.io~projected/kube-api-access-khb5q major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc9f7717-0e17-4ca2-aaa2-286a398e3435/volumes/kubernetes.io~projected/kube-api-access-4lvb4:{mountpoint:/var/lib/kubelet/pods/dc9f7717-0e17-4ca2-aaa2-286a398e3435/volumes/kubernetes.io~projected/kube-api-access-4lvb4 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f557c0f3-8e66-416c-8c4b-cebb3fd60a25/volumes/kubernetes.io~projected/kube-api-access-8vqpq:{mountpoint:/var/lib/kubelet/pods/f557c0f3-8e66-416c-8c4b-cebb3fd60a25/volumes/kubernetes.io~projected/kube-api-access-8vqpq major:0 minor:67 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/8396d26479f97379832b6b2749592eb622abe9bf0a7575dfeb928bc1917b154a/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/50539456b4bc8003b8879901cee27cf94dcb0081292f71bf399fc7de940d818c/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-108:{mountpoint:/var/lib/containers/storage/overlay/ad8ba77eba1f9171f5be59ca41b71fdea979f31770f182f1f3699be8cef8c08c/merged major:0 minor:108 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/ede19bb5ea85846b4f36ccfe86a8ffb19819b42bf0f85d1589a6cdcc8572c359/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/be6739afcfd439faf88c88a5d11f4cb52d3986406cede97e38ff331c6e16b895/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/1a6b1806e6ffe6ee5f4a90ac8df8bf236c05308f6561887b76e92a87a86d9ac3/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/6e366bfafd6c5c954e375c36e6597d94c25ce4131034518f77dbfc9cb18ca617/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/da61524f29ba06ff5369ebe440c0c8dc9debd5d8e3697c7ec8b5b8ee540674d8/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/4d8b39cc57b0e80885753607257d22a349285b7cce751e6fb1341b2a4ff8e0c4/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-141:{mountpoint:/var/lib/containers/storage/overlay/353bc030391fa049f893597644eecb066cf703cc5b3c4275d4db85d102cf88d8/merged major:0 minor:141 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/f5f58568f2b6b4576bce41062e8fb60eab65d4714544a7dbdbf8e0d9a928a563/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/52afcb9b408f6566135b40dfc056c707f90ec220e7a2f138a9c1ba7a90bf4ad1/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/8b06d31a2aacf5d0cea52b89dc8329483a7c1fe9e944d9901f570a03d7831f1c/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/4cd2f9fdfc46d2bf4a1fade67b58bf3489438c4c27dfa0b1f5e15b05f043ea5a/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/1a96d4e35e9969b1318db107e8c0b825988f593cfb9da497aabe5e51ec2fd24c/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/9915f4559e6056ecd4bd0992507938c139fec532405e27b03d55eea1b7894406/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/44f3d2edff3e1f5581c2a3f2cc5b0b4d8f7d37e8fbfbbaba15c488bca6c30239/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/fafc1b469249b72aa266805e78dda69b3e44c08bd271bd70e5e5fec7917e5ea9/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/feb30f22b082d5404b40b1e8fae03145b144e2e7c1cb686061aabfb19b166c60/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/c497451e482dd6564e591947cbc804b73dc14cb93a78b7076859b064f20192ae/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/84701bcdf2b73b35bbf31e384d493646e4a70306d9bb8c06505bc49e7b81df8f/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/11d2bb2321c179581a4e04155cb6a63b1ca4eb3e8950203848a63999c3833c71/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/63af5bc0bee2ba9ca947714642553153646a798c50645f9dcd50c903fc58cd9a/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/fe4f5cf386d9f9855340a7e9cb153dfa6eb8c1074876de35ef4a14f4cbbc0465/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/be6d62d873b6566351a499f56664671e39ae1fb43533354dba142efa2e90ce36/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/5f306505093b7cd803d1d9dcd6a9c5c20d574fb45d3c4e95898ce0cd29f7a583/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/0365be41ccc659ea9a370d6301160f362582db76b29ccdddade2f0a02361c5f1/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/5010cbecda631e571b472d1b87420def8b5e1a547cbf7ca7031d9e338f0c9c6b/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/48599c1242b1bf01f8fb62bba8dd7d4030ba27c30da9131f463469353d93c91d/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-249:{mountpoint:/var/lib/containers/storage/overlay/dd4bcf4efcc70612d9283ef65ac090d5c8e7570ba9f1d38cbeceb12126457211/merged major:0 minor:249 fsType:overlay blockSize:0} overlay_0-257:{mountpoint:/var/lib/containers/storage/overlay/581f9959b475efb74c782ef8c03ccf88d6fccfb247986d4289287d0beb601e3b/merged major:0 minor:257 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/6f3fb5be1e571c88501f3bac805e9d0b519fa11cbbefb0e342de7cb6c94c27c2/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/d998e0926db4b9bf8627b42b5775ef05b9be9a891b331c09e25ae4534adffdb4/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/b5cb260eb65f67ea82ee9f8eb1e583b9c0df490a23ca4bfc49288bd64b974f88/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/b0fc6a29a5a5c3d07a436acf52347e6613fb59c45b0363c0d7f92dd55d56e4cd/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/538c8be46d1b00ff912a7ee7bcfb5e8532d08b09a2ba2021e472a45b589b9171/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/a79fcf770ebb4489af86f64a40680fe24bc917e1464b8ee716089813819c9ce6/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/2895a3e0e477dd61697fd267efea170530a2aa63530903c48e98c5697afd2165/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-323:{mountpoint:/var/lib/containers/storage/overlay/bab773dab3764e2a5730b2029a97ac86811b6d9366c67f622b3ef9f34da2ea62/merged major:0 minor:323 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/8e35a2a661bb51ae465875d5931f9a6f0a837312b1349fc71910190b4b8755e9/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/f830d41eaf7a8669cdf430a01f805f4e45cc6e201386135a0e44fae233c1d744/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/1faad84433828f50baa716f9e58cc30acb43d497c7db96ec769b5af4bc19becd/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/577e3896c3a20ea0ead13effe8c216900a1f2027994ff055ee645f7d8a84bdd7/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/6089df2b2f5f43fb7061ef2aefa756a3c44c4451fdd96176366cffd50b48b1c7/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/3a0b94a22361dde5f6a56957cf781d4379805ab7825eb91ce44ba7b5939b0bf9/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/4dd893f7a81d4a79096bb06174a42fae338feff7a82efeac7b815eb54e14a581/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/b5fd1f9642cc8023d87cfd3d3db4fc113a33fde63a2c8cde2596f7557faff62e/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/d4d721f0532a643989f3a9103c08eaa297fd4a05ea6016bba338bed97aec13ca/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-367:{mountpoint:/var/lib/containers/storage/overlay/6a8ff4a78a95a11333fbae026a9d6256f17238f7142890377b224f8881833c2b/merged major:0 minor:367 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/cd1616e0d59c055d78b78b6bd9ea660c7641c1c2d69a218becb9d56def0e115f/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/1b7867af00fb527dea6c9f50e40d0d055aff5c63857d794865bc63e5eb11d009/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/fe71334181ac413c4440eede6dd0bb442cd0156570deb94649eadf93ea5c5546/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/3ca1dffa55765c802ff929be75ce47c18e592acc2efa7d7c5a37971d05394775/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/809de439d5283e6b57d5b6d9b1dea8570cc2462d04198e9b1337e24462b27159/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/17e4dcfe3aadf7d9425f965c5c417b19e23a6669e57ab72be3175867b616bfbc/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/f7614793283de515871adf22f1b51ab07933a0e0328a5ec980c82cfbfa822b65/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/a5c1c9350efc8733042a0eab224755bd446d4bdf85a7d05b87b47d25ef037959/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/a26442f555640b03870725e324b1c69c20504725dedb6b9573ec95897ade1d83/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/3e637e46031ff2848c6a20c3ba096cc9e5133dd859458411b865c2c329051948/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/1b693f730e218ae476c0fe59beaee57444421584c9eef772df79ceb3f7a8ae8d/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/46796476fa5f2d8fcdd74fe73dff95f444bc8260bea80ad64a7b0c1fe572239b/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/1f3ae20fa7e78233857365d5dd9a328fc893c95a3c9b512987118304d90f7cd3/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 30 17:57:09.923207 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.922529    5746 manager.go:217] Machine: {Timestamp:2026-10-30 17:57:09.921875202 +0000 UTC m=+0.088693320 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3116018 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2df2e1f4aa427798545250ddd22641 SystemUUID:ec2df2e1-f4aa-4277-9854-5250ddd22641 BootID:86d23503-89e6-498c-b24d-7d9562e4c6f3 Filesystems:[{Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:269 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:309 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/222e997a856f88fd98b37103b71fd17597db420f212199c6b2f7a93ce9b1a56d/userdata/shm DeviceMajor:0 DeviceMinor:319 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/04353ae0bb37b5944719e4306001e51e44a5b7fa7619139a861eb0146f6b33ab/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f7156936dbc3eef054d59472932ea7ad7caa7d8e53a270f84781cef90767b5c7/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b/volumes/kubernetes.io~projected/kube-api-access-fm9zl DeviceMajor:0 DeviceMinor:202 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:303 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:282 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~projected/kube-api-access-2ztqc DeviceMajor:0 DeviceMinor:296 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e69b758228cfce3f1ad8692f34b1e2661c96618a472b174269e9b95944430b64/userdata/shm DeviceMajor:0 DeviceMinor:297 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:310 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/61284ced422bbbce93df64f4c4a520ec2d52427cc95ee8ec7885a6993a7ebd53/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fca71c4819366726fe0e60c275913c31b866a2c8e051c60a82d9cad082d8faf4/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a45417c32b7d6f944c419a810f2ee280f977b8c5d434aaf220b4e76eaea45b6c/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/be8a97409d9e8d089dc721f01cb1cb97a8c2aaf56384b43057091f605a8a0525/userdata/shm DeviceMajor:0 DeviceMinor:334 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1e3c695e-3fac-49f4-838e-251b0c17fe67/volumes/kubernetes.io~projected/kube-api-access-vwf77 DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/44839f079e230e73e1588d9c1be0c51a6134dbb59817cd8dd5b3c55cadef8281/userdata/shm DeviceMajor:0 DeviceMinor:328 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~projected/kube-api-access-p5424 DeviceMajor:0 DeviceMinor:62 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1e3c695e-3fac-49f4-838e-251b0c17fe67/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:321 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a7e4861b61c44604c8b00f069806ea3ca8b46c59d3493196796b5df884133b8c/userdata/shm DeviceMajor:0 DeviceMinor:335 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:303 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f557c0f3-8e66-416c-8c4b-cebb3fd60a25/volumes/kubernetes.io~projected/kube-api-access-8vqpq DeviceMajor:0 DeviceMinor:67 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-108 DeviceMajor:0 DeviceMinor:108 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a6ee33fb-66e9-4483-b3c1-a289e9620de4/volumes/kubernetes.io~projected/kube-api-access-x42gn DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a304069c-7b5f-4ddf-9549-33797ac13658/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3/volumes/kubernetes.io~projected/kube-api-access-khb5q DeviceMajor:0 DeviceMinor:65 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/55b146394f9275007f459e946a522f4fe86e682c048c213a905141fb99fddc46/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:294 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:313 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-141 DeviceMajor:0 DeviceMinor:141 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:312 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc9f7717-0e17-4ca2-aaa2-286a398e3435/volumes/kubernetes.io~projected/kube-api-access-4lvb4 DeviceMajor:0 DeviceMinor:61 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/04e386bc-2d2b-438e-bb10-af8b05ea5b30/volumes/kubernetes.io~projected/kube-api-access-8hq7z DeviceMajor:0 DeviceMinor:66 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:308 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:311 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:315 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/892bff89c7902ee56649cc1734215455296c65b3256ddd266f83d468c6d1ce63/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a6ee33fb-66e9-4483-b3c1-a289e9620de4/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3282e78544ee7ab35e0a2d9aaeb0daf14e087e263bdccee24698a52137f4d6ae/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:286 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes/kubernetes.io~projected/kube-api-access-2z8q6 DeviceMajor:0 DeviceMinor:287 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:316 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08444539-c0d4-4b26-9bbd-c75c21b32a21/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:322 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/94750cd6b25ccb4311f07f812adc895276b1fa831e3df0a40f14703974234cd3/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~projected/kube-api-access-2jw5v DeviceMajor:0 DeviceMinor:246 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:295 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:314 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~projected/kube-api-access-28mgj DeviceMajor:0 DeviceMinor:318 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:303 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-367 DeviceMajor:0 DeviceMinor:367 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/90dddc2bf58d76d9a42310ecc46a91a129e4e43c7f28020fe20c3c12e311a499/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:203 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ba85cd81-0f4d-4907-9190-8ba2e3199aa3/volumes/kubernetes.io~projected/kube-api-access-nqhkd DeviceMajor:0 DeviceMinor:325 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:317 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-323 DeviceMajor:0 DeviceMinor:323 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:303 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab/volumes/kubernetes.io~projected/kube-api-access-r9j2g DeviceMajor:0 DeviceMinor:64 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74f8ab7cef4403c5e87a4457a33a5b13030373364ae5a1b94b0ecb34a6755af8/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ec8c10345430e69c058f5ceb1cb0f2be8da23888104b6f38b89704686c2a5a1d/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:292 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-257 DeviceMajor:0 DeviceMinor:257 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:265 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volumes/kubernetes.io~projected/kube-api-access-vc6lk DeviceMajor:0 DeviceMinor:60 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f9f5382ca9626b9391fd289f9b925080bd3059cb8255c9a462e0bdc1aacefb3e/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de40dbceb56a5e9fe4b1878df19fef2f035cbf9585edaf3e0a1f48f21b061544/userdata/shm DeviceMajor:0 DeviceMinor:254 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/762289fdc4e13abdfc08e25ba85382d613aa87b23eccdcd31cd44b7e51aebc9d/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2eeef7c4-96d5-4b07-8e8d-d8dff6385427/volumes/kubernetes.io~projected/kube-api-access-5x5tp DeviceMajor:0 DeviceMinor:270 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0aac7377-2d25-4744-8fbf-49b60ba4d2c0/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:293 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:241 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f5023e818d109e36e9626caf775d474710489b4424ef9a21e15aaf51aedef629/userdata/shm DeviceMajor:0 DeviceMinor:100 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9ca9cc19-6578-4e8d-a424-61ed63f9b562/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:245 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f/volumes/kubernetes.io~projected/kube-api-access-fdd5b DeviceMajor:0 DeviceMinor:204 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-249 DeviceMajor:0 DeviceMinor:249 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6fe540a2-725b-4723-bb8c-11b83afe9b2a/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/91a3053e-bb6f-4678-b865-4bb0e24f7dad/volumes/kubernetes.io~projected/kube-api-access-th8x7 DeviceMajor:0 DeviceMinor:63 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0c3da85c-aef5-4f3f-9381-8665d944c858/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:04353ae0bb37b59 MacAddress:0a:5a:d9:1b:c2:0a Speed:10000 Mtu:8901} {Name:222e997a856f88f MacAddress:fe:58:4a:c1:ff:a0 Speed:10000 Mtu:8901} {Name:44839f079e230e7 MacAddress:56:e0:ad:99:4c:de Speed:10000 Mtu:8901} {Name:74f8ab7cef4403c MacAddress:36:a8:aa:cf:71:b2 Speed:10000 Mtu:8901} {Name:762289fdc4e13ab MacAddress:ea:8f:00:7b:56:f4 Speed:10000 Mtu:8901} {Name:a7e4861b61c4460 MacAddress:0a:a9:5a:22:82:d5 Speed:10000 Mtu:8901} {Name:be8a97409d9e8d0 MacAddress:1a:f1:69:99:dc:88 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:ce:36:0c:89 Speed:0 Mtu:9001} {Name:br-int MacAddress:2e:40:d5:50:e5:14 Speed:0 Mtu:8901} {Name:de40dbceb56a5e9 MacAddress:26:9d:e5:3d:5c:c1 Speed:10000 Mtu:8901} {Name:e69b758228cfce3 MacAddress:0a:63:83:61:23:eb Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:ce:36:0c:89 Speed:0 Mtu:9001} {Name:f7156936dbc3eef MacAddress:be:11:bb:d6:51:df Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:02:3a:1a:ba:f6:7a Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:d2:71:b5:12:ea:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 17:57:09.923207 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923124    5746 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 17:57:09.923207 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923190    5746 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 17:57:09.923620 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923581    5746 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 17:57:09.923771 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923622    5746 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-31.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 17:57:09.923858 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923774    5746 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 17:57:09.923858 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923782    5746 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 17:57:09.923858 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923796    5746 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 17:57:09.923858 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923814    5746 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 17:57:09.923998 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.923941    5746 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 17:57:09.924156 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.924146    5746 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 17:57:09.924261 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.924252    5746 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 17:57:09.924303 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.924290    5746 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 17:57:09.924346 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.924323    5746 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 17:57:09.924391 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.924336    5746 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 17:57:09.924391 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.924386    5746 kubelet.go:405] "Adding apiserver pod source"
Oct 30 17:57:09.924481 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.924400    5746 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 17:57:09.925402 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.925387    5746 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 17:57:09.925402 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.925402    5746 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 17:57:09.926241 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.926221    5746 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 17:57:09.944051 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.943972    5746 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 17:57:09.944051 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.943990    5746 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 17:57:09.944200 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944104    5746 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 17:57:09.944324 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944311    5746 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 17:57:09.944372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944341    5746 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 17:57:09.944476 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944464    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 17:57:09.944522 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944486    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 17:57:09.944522 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944493    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 17:57:09.944522 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944498    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 17:57:09.944522 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944504    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 17:57:09.944522 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944510    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 17:57:09.944522 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944515    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 17:57:09.944522 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944520    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 17:57:09.944855 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944527    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 17:57:09.944855 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944533    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 17:57:09.944855 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944547    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 17:57:09.944855 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944558    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 17:57:09.944855 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944612    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 17:57:09.944855 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.944618    5746 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 17:57:09.948165 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.948149    5746 server.go:1263] "Started kubelet"
Oct 30 17:57:09.949253 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.948259    5746 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 17:57:09.949253 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.948495    5746 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 17:57:09.949253 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.949167    5746 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 17:57:09.948862 ip-10-0-131-31 systemd[1]: Started Kubernetes Kubelet.
Oct 30 17:57:09.949621 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.949582    5746 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 17:57:09.955578 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.955558    5746 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 17:57:09.955734 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.955605    5746 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 17:57:09.956091 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.955605    5746 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 17:57:09.957150 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:09.957125    5746 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-31.ec2.internal\" not found"
Oct 30 17:57:09.957296 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.957194    5746 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 17:57:09.957296 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.957203    5746 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 17:57:09.957421 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.957299    5746 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 17:50:59 +0000 UTC" deadline="2028-07-14 23:02:16.252438862 +0000 UTC"
Oct 30 17:57:09.957421 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.957324    5746 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14957h5m6.295117826s"
Oct 30 17:57:09.957531 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.957510    5746 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 17:57:09.970924 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.964652    5746 factory.go:153] Registering CRI-O factory
Oct 30 17:57:09.970924 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.964673    5746 factory.go:223] Registration of the crio container factory successfully
Oct 30 17:57:09.970924 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.964729    5746 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 17:57:09.970924 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.964749    5746 factory.go:55] Registering systemd factory
Oct 30 17:57:09.970924 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.964758    5746 factory.go:223] Registration of the systemd container factory successfully
Oct 30 17:57:09.970924 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.964779    5746 factory.go:103] Registering Raw factory
Oct 30 17:57:09.970924 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.964793    5746 manager.go:1199] Started watching for new ooms in manager
Oct 30 17:57:09.970924 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.965146    5746 manager.go:319] Starting recovery of all containers
Oct 30 17:57:09.974391 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.974325    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" volumeName="kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-binary-copy" seLinuxMountContext=""
Oct 30 17:57:09.974544 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.974524    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aac7377-2d25-4744-8fbf-49b60ba4d2c0" volumeName="kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-metrics-client-ca" seLinuxMountContext=""
Oct 30 17:57:09.974642 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.974628    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 17:57:09.974726 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.974712    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc9f7717-0e17-4ca2-aaa2-286a398e3435" volumeName="kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-daemon-config" seLinuxMountContext=""
Oct 30 17:57:09.974808 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.974794    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc9f7717-0e17-4ca2-aaa2-286a398e3435" volumeName="kubernetes.io/projected/dc9f7717-0e17-4ca2-aaa2-286a398e3435-kube-api-access-4lvb4" seLinuxMountContext=""
Oct 30 17:57:09.974923 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.974892    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f557c0f3-8e66-416c-8c4b-cebb3fd60a25" volumeName="kubernetes.io/projected/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-kube-api-access-8vqpq" seLinuxMountContext=""
Oct 30 17:57:09.975027 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975010    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e3c695e-3fac-49f4-838e-251b0c17fe67" volumeName="kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs" seLinuxMountContext=""
Oct 30 17:57:09.975117 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975099    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 17:57:09.975206 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975191    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ca9cc19-6578-4e8d-a424-61ed63f9b562" volumeName="kubernetes.io/configmap/9ca9cc19-6578-4e8d-a424-61ed63f9b562-metrics-client-ca" seLinuxMountContext=""
Oct 30 17:57:09.975287 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975275    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f" volumeName="kubernetes.io/secret/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-cert" seLinuxMountContext=""
Oct 30 17:57:09.975365 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975352    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08444539-c0d4-4b26-9bbd-c75c21b32a21" volumeName="kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret" seLinuxMountContext=""
Oct 30 17:57:09.975439 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975428    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aac7377-2d25-4744-8fbf-49b60ba4d2c0" volumeName="kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 17:57:09.975539 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975524    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aac7377-2d25-4744-8fbf-49b60ba4d2c0" volumeName="kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 17:57:09.975623 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975609    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c3da85c-aef5-4f3f-9381-8665d944c858" volumeName="kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 17:57:09.975719 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975704    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2eeef7c4-96d5-4b07-8e8d-d8dff6385427" volumeName="kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-tls" seLinuxMountContext=""
Oct 30 17:57:09.975819 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975786    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fe540a2-725b-4723-bb8c-11b83afe9b2a" volumeName="kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-tuned" seLinuxMountContext=""
Oct 30 17:57:09.975947 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.975931    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 17:57:09.976049 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976032    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ffa15d5-c851-4b6e-bfc7-faa42f332f0b" volumeName="kubernetes.io/projected/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-api-access-fm9zl" seLinuxMountContext=""
Oct 30 17:57:09.976175 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976151    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 17:57:09.976268 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976254    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6ee33fb-66e9-4483-b3c1-a289e9620de4" volumeName="kubernetes.io/configmap/a6ee33fb-66e9-4483-b3c1-a289e9620de4-config-volume" seLinuxMountContext=""
Oct 30 17:57:09.976349 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976335    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" volumeName="kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 17:57:09.976431 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976417    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04e386bc-2d2b-438e-bb10-af8b05ea5b30" volumeName="kubernetes.io/projected/04e386bc-2d2b-438e-bb10-af8b05ea5b30-kube-api-access-8hq7z" seLinuxMountContext=""
Oct 30 17:57:09.976522 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976508    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aac7377-2d25-4744-8fbf-49b60ba4d2c0" volumeName="kubernetes.io/projected/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-kube-api-access-2ztqc" seLinuxMountContext=""
Oct 30 17:57:09.976611 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976591    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c43e52-8525-45ad-9af8-026b029f538b" volumeName="kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert" seLinuxMountContext=""
Oct 30 17:57:09.976692 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976680    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls" seLinuxMountContext=""
Oct 30 17:57:09.976767 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976754    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91a3053e-bb6f-4678-b865-4bb0e24f7dad" volumeName="kubernetes.io/configmap/91a3053e-bb6f-4678-b865-4bb0e24f7dad-serviceca" seLinuxMountContext=""
Oct 30 17:57:09.976851 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976839    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6ee33fb-66e9-4483-b3c1-a289e9620de4" volumeName="kubernetes.io/secret/a6ee33fb-66e9-4483-b3c1-a289e9620de4-metrics-tls" seLinuxMountContext=""
Oct 30 17:57:09.976965 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.976947    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2eeef7c4-96d5-4b07-8e8d-d8dff6385427" volumeName="kubernetes.io/projected/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-kube-api-access-5x5tp" seLinuxMountContext=""
Oct 30 17:57:09.977085 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977070    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c43e52-8525-45ad-9af8-026b029f538b" volumeName="kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert" seLinuxMountContext=""
Oct 30 17:57:09.977375 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977351    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 17:57:09.977417 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977380    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ca9cc19-6578-4e8d-a424-61ed63f9b562" volumeName="kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 17:57:09.977417 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977394    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ca9cc19-6578-4e8d-a424-61ed63f9b562" volumeName="kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 17:57:09.977417 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977406    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c3da85c-aef5-4f3f-9381-8665d944c858" volumeName="kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-config" seLinuxMountContext=""
Oct 30 17:57:09.977510 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977419    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c43e52-8525-45ad-9af8-026b029f538b" volumeName="kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config" seLinuxMountContext=""
Oct 30 17:57:09.977510 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977430    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 17:57:09.977510 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977442    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 17:57:09.977510 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977453    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a304069c-7b5f-4ddf-9549-33797ac13658" volumeName="kubernetes.io/configmap/a304069c-7b5f-4ddf-9549-33797ac13658-konnectivity-ca" seLinuxMountContext=""
Oct 30 17:57:09.977510 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977464    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aac7377-2d25-4744-8fbf-49b60ba4d2c0" volumeName="kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-federate-client-tls" seLinuxMountContext=""
Oct 30 17:57:09.977510 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977475    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c3da85c-aef5-4f3f-9381-8665d944c858" volumeName="kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-env-overrides" seLinuxMountContext=""
Oct 30 17:57:09.977510 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977496    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f" volumeName="kubernetes.io/projected/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-kube-api-access-fdd5b" seLinuxMountContext=""
Oct 30 17:57:09.977510 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977506    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2eeef7c4-96d5-4b07-8e8d-d8dff6385427" volumeName="kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-metrics-client-ca" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977518    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2eeef7c4-96d5-4b07-8e8d-d8dff6385427" volumeName="kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977529    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c43e52-8525-45ad-9af8-026b029f538b" volumeName="kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977547    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c43e52-8525-45ad-9af8-026b029f538b" volumeName="kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977561    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977574    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ffa15d5-c851-4b6e-bfc7-faa42f332f0b" volumeName="kubernetes.io/empty-dir/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-data-volume" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977585    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ffa15d5-c851-4b6e-bfc7-faa42f332f0b" volumeName="kubernetes.io/secret/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977596    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977609    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977621    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91a3053e-bb6f-4678-b865-4bb0e24f7dad" volumeName="kubernetes.io/projected/91a3053e-bb6f-4678-b865-4bb0e24f7dad-kube-api-access-th8x7" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977632    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba85cd81-0f4d-4907-9190-8ba2e3199aa3" volumeName="kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977644    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc9f7717-0e17-4ca2-aaa2-286a398e3435" volumeName="kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cni-binary-copy" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977655    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f557c0f3-8e66-416c-8c4b-cebb3fd60a25" volumeName="kubernetes.io/configmap/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-iptables-alerter-script" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977666    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fe540a2-725b-4723-bb8c-11b83afe9b2a" volumeName="kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-tmp" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977678    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" volumeName="kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977689    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977710    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977723    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6ee33fb-66e9-4483-b3c1-a289e9620de4" volumeName="kubernetes.io/empty-dir/a6ee33fb-66e9-4483-b3c1-a289e9620de4-tmp-dir" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977732    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" volumeName="kubernetes.io/projected/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-kube-api-access-khb5q" seLinuxMountContext=""
Oct 30 17:57:09.977742 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977740    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04e386bc-2d2b-438e-bb10-af8b05ea5b30" volumeName="kubernetes.io/empty-dir/04e386bc-2d2b-438e-bb10-af8b05ea5b30-tmp-dir" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977748    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c3da85c-aef5-4f3f-9381-8665d944c858" volumeName="kubernetes.io/projected/0c3da85c-aef5-4f3f-9381-8665d944c858-kube-api-access-vc6lk" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977760    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fe540a2-725b-4723-bb8c-11b83afe9b2a" volumeName="kubernetes.io/projected/6fe540a2-725b-4723-bb8c-11b83afe9b2a-kube-api-access-p5424" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977767    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ca9cc19-6578-4e8d-a424-61ed63f9b562" volumeName="kubernetes.io/projected/9ca9cc19-6578-4e8d-a424-61ed63f9b562-kube-api-access-2jw5v" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977775    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a304069c-7b5f-4ddf-9549-33797ac13658" volumeName="kubernetes.io/secret/a304069c-7b5f-4ddf-9549-33797ac13658-agent-certs" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977783    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2eeef7c4-96d5-4b07-8e8d-d8dff6385427" volumeName="kubernetes.io/empty-dir/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-textfile" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977790    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c43e52-8525-45ad-9af8-026b029f538b" volumeName="kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977797    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c43e52-8525-45ad-9af8-026b029f538b" volumeName="kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977805    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977812    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977820    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977827    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977835    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aac7377-2d25-4744-8fbf-49b60ba4d2c0" volumeName="kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-client-tls" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977842    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50c43e52-8525-45ad-9af8-026b029f538b" volumeName="kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977849    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977856    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aac7377-2d25-4744-8fbf-49b60ba4d2c0" volumeName="kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977864    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ffa15d5-c851-4b6e-bfc7-faa42f332f0b" volumeName="kubernetes.io/configmap/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977873    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab" volumeName="kubernetes.io/projected/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kube-api-access-r9j2g" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977879    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0aac7377-2d25-4744-8fbf-49b60ba4d2c0" volumeName="kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977887    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c3da85c-aef5-4f3f-9381-8665d944c858" volumeName="kubernetes.io/secret/0c3da85c-aef5-4f3f-9381-8665d944c858-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977894    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e3c695e-3fac-49f4-838e-251b0c17fe67" volumeName="kubernetes.io/projected/1e3c695e-3fac-49f4-838e-251b0c17fe67-kube-api-access-vwf77" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977901    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2eeef7c4-96d5-4b07-8e8d-d8dff6385427" volumeName="kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977937    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977949    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c9e9544-bb16-400e-b614-f06894ad09da" volumeName="kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977961    5746 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6ee33fb-66e9-4483-b3c1-a289e9620de4" volumeName="kubernetes.io/projected/a6ee33fb-66e9-4483-b3c1-a289e9620de4-kube-api-access-x42gn" seLinuxMountContext=""
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977972    5746 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 17:57:09.978665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.977979    5746 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 17:57:09.984409 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.984352    5746 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:57:09.992131 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:09.992095    5746 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 17:57:10.002539 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.002517    5746 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 17:57:10.002620 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.002542    5746 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 17:57:10.002620 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.002566    5746 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 17:57:10.002681 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:10.002615    5746 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 17:57:10.017977 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.017894    5746 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 17:57:10.019682 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.019653    5746 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:57:10.021377 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.021328    5746 generic.go:356] "Generic (PLEG): container finished" podID="8158fd6db0c3186f36b1583fbfe4e93d" containerID="580ea1c67f42aded753ef10d011b6fb5a5503c09d3b20688b555dc4c96367f7f" exitCode=0
Oct 30 17:57:10.030353 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.030327    5746 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:57:10.030745 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.030430    5746 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 17:57:10.036202 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.036183    5746 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="e946c081458c1ce0cbe6c9d8af6ccfb16ad9506ebb43cd622b76d2bcfff69a85" exitCode=0
Oct 30 17:57:10.036319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.036309    5746 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="ac3f698457feeb271bc772ec1c42995c71de66f9b8c4229290143410044e7e3b" exitCode=0
Oct 30 17:57:10.036388 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.036380    5746 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="64d8d09702db5b9a5592ef8cb793db6595ddef1b8d27b86426a53f2b35be981f" exitCode=0
Oct 30 17:57:10.036436 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.036429    5746 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="69b3ccc672b3f1486f7610dedae2ab1694b19d2e531f5d0b0439e86836c91916" exitCode=0
Oct 30 17:57:10.036490 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.036479    5746 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="924cbf10b3686ae01735a70cc88c5ab217549ea641730075743ac07fb1915e84" exitCode=0
Oct 30 17:57:10.036558 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.036547    5746 generic.go:356] "Generic (PLEG): container finished" podID="d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3" containerID="f72a7876e6b80ffe3d1675c304d9b956040506ab8cc6afc6ac61f751117112ba" exitCode=0
Oct 30 17:57:10.041404 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.040975    5746 generic.go:356] "Generic (PLEG): container finished" podID="7c9e9544-bb16-400e-b614-f06894ad09da" containerID="b75756a6b05f4a5b01290d9a5a86376e4d82c40388d9b932e64cf8029d3fd207" exitCode=0
Oct 30 17:57:10.081485 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.081445    5746 generic.go:356] "Generic (PLEG): container finished" podID="2eeef7c4-96d5-4b07-8e8d-d8dff6385427" containerID="4f9fb162b9d50e20c155d81a82a737135fd2512061bcaebb26e5a5928c00bbe1" exitCode=0
Oct 30 17:57:10.096954 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.096931    5746 manager.go:324] Recovery completed
Oct 30 17:57:10.102719 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:10.102684    5746 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 17:57:10.123995 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.123966    5746 cpu_manager.go:226] "Starting" policy="none"
Oct 30 17:57:10.123995 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.123988    5746 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 17:57:10.124163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124008    5746 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 17:57:10.124213 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124164    5746 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 17:57:10.124213 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124179    5746 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 17:57:10.124213 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124202    5746 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 17:57:10.124213 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124210    5746 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 17:57:10.124333 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124217    5746 policy_none.go:50] "Start"
Oct 30 17:57:10.124333 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124228    5746 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 17:57:10.124333 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124241    5746 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 17:57:10.124436 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124423    5746 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 17:57:10.124469 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124439    5746 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 17:57:10.124469 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.124448    5746 policy_none.go:44] "Start"
Oct 30 17:57:10.129813 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.129245    5746 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 17:57:10.135848 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.135831    5746 manager.go:341] "Starting Device Plugin manager"
Oct 30 17:57:10.135968 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:10.135861    5746 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 17:57:10.135968 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.135873    5746 server.go:87] "Starting device plugin registration server"
Oct 30 17:57:10.136206 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.136190    5746 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 17:57:10.136318 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.136206    5746 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 17:57:10.136318 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.136286    5746 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 17:57:10.136415 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.136376    5746 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 17:57:10.136458 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.136421    5746 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 17:57:10.136458 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.136428    5746 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 17:57:10.137037 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.136483    5746 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T17:57:10.136406212Z","UUID":"293e102d-e292-4fb8-8d08-0fea4f7be602","Handler":null,"Name":"","Endpoint":""}
Oct 30 17:57:10.138388 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:10.138352    5746 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 17:57:10.138526 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.138505    5746 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 17:57:10.138627 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.138567    5746 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 17:57:10.236902 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.236873    5746 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 17:57:10.239152 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.239130    5746 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 17:57:10.239283 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.239170    5746 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 17:57:10.239283 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.239187    5746 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-31.ec2.internal" event="NodeHasSufficientPID"
Oct 30 17:57:10.239283 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.239246    5746 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.277398 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.277327    5746 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.277521 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.277404    5746 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.303385 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.303333    5746 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"]
Oct 30 17:57:10.304112 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.304092    5746 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.304112 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.304092    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal" event={"ID":"8158fd6db0c3186f36b1583fbfe4e93d","Type":"ContainerStarted","Data":"e16eccafccea1c053ce0317a6afb0f27fc643a675579860ac8964bb295f973aa"}
Oct 30 17:57:10.304332 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.304138    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal" event={"ID":"8158fd6db0c3186f36b1583fbfe4e93d","Type":"ContainerDied","Data":"580ea1c67f42aded753ef10d011b6fb5a5503c09d3b20688b555dc4c96367f7f"}
Oct 30 17:57:10.304332 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.304159    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal" event={"ID":"8158fd6db0c3186f36b1583fbfe4e93d","Type":"ContainerStarted","Data":"94750cd6b25ccb4311f07f812adc895276b1fa831e3df0a40f14703974234cd3"}
Oct 30 17:57:10.304332 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.304207    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal" event={"ID":"167496dc01702046817db5cdac7ef75c","Type":"ContainerStarted","Data":"fca0eba0397f131174ac8108fb99fa0f85286f6053178a4c2980961e04326f1e"}
Oct 30 17:57:10.304332 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.304226    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal" event={"ID":"167496dc01702046817db5cdac7ef75c","Type":"ContainerStarted","Data":"61284ced422bbbce93df64f4c4a520ec2d52427cc95ee8ec7885a6993a7ebd53"}
Oct 30 17:57:10.304531 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.304358    5746 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.331354 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.331329    5746 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 17:57:10.331486 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:10.331387    5746 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-31.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.331954 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.331938    5746 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 17:57:10.332092 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:10.332069    5746 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.422081 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.422045    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.422409 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.422388    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.422565 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.422547    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167496dc01702046817db5cdac7ef75c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-31.ec2.internal\" (UID: \"167496dc01702046817db5cdac7ef75c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.523348 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.523311    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.523348 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.523353    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.523573 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.523376    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167496dc01702046817db5cdac7ef75c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-31.ec2.internal\" (UID: \"167496dc01702046817db5cdac7ef75c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.523573 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.523433    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.523573 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.523433    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8158fd6db0c3186f36b1583fbfe4e93d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" (UID: \"8158fd6db0c3186f36b1583fbfe4e93d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.523573 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.523425    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/167496dc01702046817db5cdac7ef75c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-31.ec2.internal\" (UID: \"167496dc01702046817db5cdac7ef75c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:10.924953 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.924895    5746 apiserver.go:52] "Watching apiserver"
Oct 30 17:57:10.982703 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.982669    5746 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 17:57:10.983330 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:10.983308    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lvffz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal","openshift-multus/multus-7zqrb","openshift-multus/multus-additional-cni-plugins-lfmxr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk","openshift-dns/node-resolver-lxg8q","openshift-insights/insights-runtime-extractor-zwh9d","openshift-monitoring/node-exporter-z7x2c","openshift-console/console-64557c6d97-hgqx4","openshift-ovn-kubernetes/ovnkube-node-krkmg","openshift-monitoring/prometheus-k8s-0","openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7","openshift-network-operator/iptables-alerter-6s62t","kube-system/konnectivity-agent-s6687","openshift-cluster-node-tuning-operator/tuned-hmlkj","openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n","kube-system/global-pull-secret-syncer-trnpq","openshift-image-registry/node-ca-vxvtw","openshift-network-diagnostics/network-check-target-d9ml4","openshift-dns/dns-default-2ljs5","openshift-multus/network-metrics-daemon-dttj5"]
Oct 30 17:57:11.000893 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.000857    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 17:57:11.000893 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.000857    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v7j5r\""
Oct 30 17:57:11.001058 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.000932    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pc6fd\""
Oct 30 17:57:11.001502 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.001490    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 17:57:11.001652 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.001637    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mmrm7\""
Oct 30 17:57:11.001702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.001679    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-8vdz2\""
Oct 30 17:57:11.002215 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.002202    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z9xqn\""
Oct 30 17:57:11.002253 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.002234    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-26whl\""
Oct 30 17:57:11.003052 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003036    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.003126 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003046    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 17:57:11.003353 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003332    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.003353 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003334    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 17:57:11.003497 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003422    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 17:57:11.003497 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003425    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.003497 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003468    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.003614 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003528    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.003614 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003544    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 17:57:11.003614 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003565    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.003614 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003578    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.003614 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003598    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 17:57:11.003614 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003599    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.003824 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003599    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 17:57:11.003824 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003647    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 17:57:11.003824 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003667    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.003824 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003671    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.003824 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003682    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.003824 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.003720    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 17:57:11.014801 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.014787    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.015005 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.014992    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dxtgs\""
Oct 30 17:57:11.015863 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.015849    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 17:57:11.016071 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.016058    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2dl6z\""
Oct 30 17:57:11.016371 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.016358    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 17:57:11.016484 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.016472    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.016520 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.016511    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 17:57:11.016584 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.016573    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.016732 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.016718    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.016788 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.016723    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 17:57:11.016788 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.016737    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.017201 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017188    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 17:57:11.017238 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017212    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksmnn\""
Oct 30 17:57:11.017267 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017246    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 17:57:11.017311 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017287    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 17:57:11.017577 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017565    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.017867 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017853    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.017937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017877    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.017988 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017948    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 17:57:11.017988 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017958    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 17:57:11.018064 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.017992    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 17:57:11.018311 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.018298    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.018526 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.018513    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-tzlmk\""
Oct 30 17:57:11.018574 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.018514    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 17:57:11.018701 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.018687    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-h85wj\""
Oct 30 17:57:11.018745 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.018729    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 17:57:11.018895 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.018882    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-98z5p\""
Oct 30 17:57:11.018970 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.018882    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 17:57:11.028320 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.028298    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 17:57:11.029669 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.029651    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 17:57:11.029737 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.029717    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vkdz6\""
Oct 30 17:57:11.029773 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.029745    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 17:57:11.029817 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.029745    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 17:57:11.059646 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.059616    5746 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 17:57:11.085229 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.085208    5746 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:11.085375 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.085361    5746 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:11.101097 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101054    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 17:57:11.101097 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101077    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 17:57:11.101239 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101185    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 17:57:11.101282 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101270    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 17:57:11.101353 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101309    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 17:57:11.101353 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101331    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 17:57:11.101464 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101387    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 17:57:11.101464 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101400    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 17:57:11.101544 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101494    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 17:57:11.101544 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101495    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 17:57:11.101672 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101602    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 17:57:11.101672 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101623    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 17:57:11.101672 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101640    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 17:57:11.101820 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101668    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 17:57:11.101820 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101684    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zxlb9\""
Oct 30 17:57:11.101820 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101675    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-z6vc6\""
Oct 30 17:57:11.101820 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101709    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 17:57:11.101820 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101775    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hqs45\""
Oct 30 17:57:11.101820 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101776    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 17:57:11.101820 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101784    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7j6rv\""
Oct 30 17:57:11.101820 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101820    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-69ncnlhdiul6u\""
Oct 30 17:57:11.102114 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101823    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bkbt2\""
Oct 30 17:57:11.102114 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.101836    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cclgx\""
Oct 30 17:57:11.102177 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.102145    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 17:57:11.102230 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.102215    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 17:57:11.102553 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.102539    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 17:57:11.102603 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.102570    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 17:57:11.102883 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.102869    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 17:57:11.102947 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.102869    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 17:57:11.108123 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.108105    5746 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 17:57:11.108176 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.108149    5746 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-31.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:11.108746 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.108734    5746 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 17:57:11.108777 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.108767    5746 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-31.ec2.internal"
Oct 30 17:57:11.125133 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125113    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-system-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.125198 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125136    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-k8s-cni-cncf-io\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.125198 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125154    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-bin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.125198 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125170    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a3053e-bb6f-4678-b865-4bb0e24f7dad-host\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:57:11.125317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125219    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.125317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125244    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-kubelet\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.125317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125261    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-etc-kubernetes\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.125317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125281    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.125448 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125407    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-iptables-alerter-script\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:57:11.125480 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125444    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-metrics-client-ca\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.125556 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125528    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cnibin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.125657 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.125568    5746 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:11.125657 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125592    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.125657 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125621    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-multus\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.125792 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125663    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-iptables-alerter-script\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:57:11.125792 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125692    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-metrics-client-ca\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.125792 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.125727    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:11.625702837 +0000 UTC m=+1.792520954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:11.125792 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125753    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.125792 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125785    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdd5b\" (UniqueName: \"kubernetes.io/projected/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-kube-api-access-fdd5b\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:57:11.126076 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125817    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysconfig\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.126076 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125845    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a304069c-7b5f-4ddf-9549-33797ac13658-agent-certs\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:57:11.126076 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125879    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-wtmp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.126076 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.125950    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.126076 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126047    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9j2g\" (UniqueName: \"kubernetes.io/projected/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kube-api-access-r9j2g\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.126076 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126060    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.126076 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126077    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-crio-socket\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.126382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126096    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-tls\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.126382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126113    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.126382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126152    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a304069c-7b5f-4ddf-9549-33797ac13658-agent-certs\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:57:11.126382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126230    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.126382 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.126246    5746 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:11.126382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126317    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-sys-fs\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.126382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126337    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-run\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126397    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hq7z\" (UniqueName: \"kubernetes.io/projected/04e386bc-2d2b-438e-bb10-af8b05ea5b30-kube-api-access-8hq7z\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126415    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-tls\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.126427    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:11.626416648 +0000 UTC m=+1.793234753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126458    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-metrics-client-ca\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126475    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126490    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-systemd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126503    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.126544    5746 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126544    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.126565    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:11.626559104 +0000 UTC m=+1.793377210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126585    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-registration-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126617    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-cert\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126644    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a304069c-7b5f-4ddf-9549-33797ac13658-konnectivity-ca\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126674    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126686    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-metrics-client-ca\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.126702 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126703    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c3da85c-aef5-4f3f-9381-8665d944c858-ovn-node-metrics-cert\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126730    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126756    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-serving-certs-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126781    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126806    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28mgj\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126814    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-cert\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126833    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-lib-modules\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126848    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a304069c-7b5f-4ddf-9549-33797ac13658-konnectivity-ca\") pod \"konnectivity-agent-s6687\" (UID: \"a304069c-7b5f-4ddf-9549-33797ac13658\") " pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126859    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126882    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-etc-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126902    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126938    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126954    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-env-overrides\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.126954    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c3da85c-aef5-4f3f-9381-8665d944c858-ovn-node-metrics-cert\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127004    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127014    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-conf\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127004    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-serving-certs-ca-bundle\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127043    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cni-binary-copy\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127078    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-env-overrides\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127084    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e386bc-2d2b-438e-bb10-af8b05ea5b30-tmp-dir\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127127    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127154    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04e386bc-2d2b-438e-bb10-af8b05ea5b30-tmp-dir\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127194    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cni-binary-copy\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127204    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127308    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127324    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.127319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127329    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127352    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127356    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a3053e-bb6f-4678-b865-4bb0e24f7dad-serviceca\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127473    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ee33fb-66e9-4483-b3c1-a289e9620de4-config-volume\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127507    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127529    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-tuned\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127556    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-var-lib-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127582    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127605    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-sys\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127605    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08444539-c0d4-4b26-9bbd-c75c21b32a21-original-pull-secret\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127609    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-tuned\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127637    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127677    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ee33fb-66e9-4483-b3c1-a289e9620de4-config-volume\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127671    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127713    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127792    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-telemeter-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127797    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th8x7\" (UniqueName: \"kubernetes.io/projected/91a3053e-bb6f-4678-b865-4bb0e24f7dad-kube-api-access-th8x7\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127823    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ca9cc19-6578-4e8d-a424-61ed63f9b562-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127838    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127873    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-var-lib-kubelet\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127843    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127939    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a3053e-bb6f-4678-b865-4bb0e24f7dad-serviceca\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127952    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.127980    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-netd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128031    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128063    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128082    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128035    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128094    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc6lk\" (UniqueName: \"kubernetes.io/projected/0c3da85c-aef5-4f3f-9381-8665d944c858-kube-api-access-vc6lk\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128126    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ca9cc19-6578-4e8d-a424-61ed63f9b562-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128139    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128155    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ztqc\" (UniqueName: \"kubernetes.io/projected/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-kube-api-access-2ztqc\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128183    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128210    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128240    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128238    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-multus-certs\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128297    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128304    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ca9cc19-6578-4e8d-a424-61ed63f9b562-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:57:11.128382 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128388    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x5tp\" (UniqueName: \"kubernetes.io/projected/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-kube-api-access-5x5tp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128420    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128448    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-host-slash\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128465    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128467    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.128485    5746 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128473    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128521    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.128555    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:11.62854151 +0000 UTC m=+1.795359615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"service-ca" not registered
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128575    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128604    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-root\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128630    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-kubelet\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128660    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128685    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-socket-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128710    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-dbus\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128827    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-data-volume\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128872    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128895    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128937    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5424\" (UniqueName: \"kubernetes.io/projected/6fe540a2-725b-4723-bb8c-11b83afe9b2a-kube-api-access-p5424\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128949    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-data-volume\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128961    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.128992    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-slash\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129013    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129029    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129043    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129058    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-kubernetes\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129072    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-hostroot\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129086    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-conf-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129103    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lvb4\" (UniqueName: \"kubernetes.io/projected/dc9f7717-0e17-4ca2-aaa2-286a398e3435-kube-api-access-4lvb4\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129118    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129126    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e3c695e-3fac-49f4-838e-251b0c17fe67-metrics-certs\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129134    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9zl\" (UniqueName: \"kubernetes.io/projected/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-api-access-fm9zl\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129150    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-modprobe-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129173    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-host\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129273    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129302    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129310    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129312    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6ee33fb-66e9-4483-b3c1-a289e9620de4-metrics-tls\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129337    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-textfile\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129361    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-netns\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129374    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-textfile\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129381    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-log-socket\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129398    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-config\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129414    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-sys\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129429    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04e386bc-2d2b-438e-bb10-af8b05ea5b30-hosts-file\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129456    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129468    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6ee33fb-66e9-4483-b3c1-a289e9620de4-metrics-tls\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129476    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-bin\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129497    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-federate-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129531    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-config\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129603    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-kubelet-config\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129626    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jw5v\" (UniqueName: \"kubernetes.io/projected/9ca9cc19-6578-4e8d-a424-61ed63f9b562-kube-api-access-2jw5v\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129653    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129676    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-device-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.129705    5746 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.129760    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:11.629744938 +0000 UTC m=+1.796563061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129704    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x42gn\" (UniqueName: \"kubernetes.io/projected/a6ee33fb-66e9-4483-b3c1-a289e9620de4-kube-api-access-x42gn\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129772    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-federate-client-tls\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129870    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cnibin\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129919    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-netns\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129931    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129950    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.129971    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130002    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-socket-dir-parent\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130041    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-daemon-config\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130143    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-ovn\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130228    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130228    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130241    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130283    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-daemon-config\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130283    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130312    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee33fb-66e9-4483-b3c1-a289e9620de4-tmp-dir\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130330    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130349    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-systemd-units\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130365    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vqpq\" (UniqueName: \"kubernetes.io/projected/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-kube-api-access-8vqpq\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130403    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.130415    5746 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 17:57:11.130372 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130422    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.130456    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:11.630445696 +0000 UTC m=+1.797263802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-config" not registered
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130475    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khb5q\" (UniqueName: \"kubernetes.io/projected/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-kube-api-access-khb5q\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130503    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130478    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee33fb-66e9-4483-b3c1-a289e9620de4-tmp-dir\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130527    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-tmp\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130569    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-binary-copy\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130595    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130608    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe540a2-725b-4723-bb8c-11b83afe9b2a-tmp\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130621    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130648    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-systemd\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130656    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130674    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-node-log\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130718    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-script-lib\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130742    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cni-binary-copy\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130762    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130763    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130812    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130853    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwf77\" (UniqueName: \"kubernetes.io/projected/1e3c695e-3fac-49f4-838e-251b0c17fe67-kube-api-access-vwf77\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130896    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c3da85c-aef5-4f3f-9381-8665d944c858-ovnkube-script-lib\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130938    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.130987    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.134223 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.131134    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.135296 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.134796    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.148731 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.148709    5746 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 17:57:11.148731 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.148726    5746 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 17:57:11.148731 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.148735    5746 projected.go:196] Error preparing data for projected volume kube-api-access-2z8q6 for pod openshift-console/console-64557c6d97-hgqx4: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:11.148976 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.148779    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6 podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:11.648768377 +0000 UTC m=+1.815586516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2z8q6" (UniqueName: "kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:11.148976 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.148960    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hq7z\" (UniqueName: \"kubernetes.io/projected/04e386bc-2d2b-438e-bb10-af8b05ea5b30-kube-api-access-8hq7z\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:57:11.148976 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.148968    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9j2g\" (UniqueName: \"kubernetes.io/projected/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kube-api-access-r9j2g\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.149246 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.149225    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdd5b\" (UniqueName: \"kubernetes.io/projected/2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f-kube-api-access-fdd5b\") pod \"ingress-canary-lvffz\" (UID: \"2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f\") " pod="openshift-ingress-canary/ingress-canary-lvffz"
Oct 30 17:57:11.157662 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157180    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9zl\" (UniqueName: \"kubernetes.io/projected/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-kube-api-access-fm9zl\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.157662 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157253    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x5tp\" (UniqueName: \"kubernetes.io/projected/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-kube-api-access-5x5tp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.157662 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157282    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x42gn\" (UniqueName: \"kubernetes.io/projected/a6ee33fb-66e9-4483-b3c1-a289e9620de4-kube-api-access-x42gn\") pod \"dns-default-2ljs5\" (UID: \"a6ee33fb-66e9-4483-b3c1-a289e9620de4\") " pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:11.157662 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157310    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th8x7\" (UniqueName: \"kubernetes.io/projected/91a3053e-bb6f-4678-b865-4bb0e24f7dad-kube-api-access-th8x7\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:57:11.157662 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157337    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lvb4\" (UniqueName: \"kubernetes.io/projected/dc9f7717-0e17-4ca2-aaa2-286a398e3435-kube-api-access-4lvb4\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.157662 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157463    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwf77\" (UniqueName: \"kubernetes.io/projected/1e3c695e-3fac-49f4-838e-251b0c17fe67-kube-api-access-vwf77\") pod \"network-metrics-daemon-dttj5\" (UID: \"1e3c695e-3fac-49f4-838e-251b0c17fe67\") " pod="openshift-multus/network-metrics-daemon-dttj5"
Oct 30 17:57:11.157662 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157506    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jw5v\" (UniqueName: \"kubernetes.io/projected/9ca9cc19-6578-4e8d-a424-61ed63f9b562-kube-api-access-2jw5v\") pod \"prometheus-operator-9cd89b74c-5bn4n\" (UID: \"9ca9cc19-6578-4e8d-a424-61ed63f9b562\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-5bn4n"
Oct 30 17:57:11.157928 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157723    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc6lk\" (UniqueName: \"kubernetes.io/projected/0c3da85c-aef5-4f3f-9381-8665d944c858-kube-api-access-vc6lk\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.157928 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.157859    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ztqc\" (UniqueName: \"kubernetes.io/projected/0aac7377-2d25-4744-8fbf-49b60ba4d2c0-kube-api-access-2ztqc\") pod \"telemeter-client-69ffd7f44c-6nfl7\" (UID: \"0aac7377-2d25-4744-8fbf-49b60ba4d2c0\") " pod="openshift-monitoring/telemeter-client-69ffd7f44c-6nfl7"
Oct 30 17:57:11.158093 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.158073    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28mgj\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj\") pod \"prometheus-k8s-0\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.158127 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.158076    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqhkd\" (UniqueName: \"kubernetes.io/projected/ba85cd81-0f4d-4907-9190-8ba2e3199aa3-kube-api-access-nqhkd\") pod \"network-check-target-d9ml4\" (UID: \"ba85cd81-0f4d-4907-9190-8ba2e3199aa3\") " pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:57:11.161708 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.161684    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vqpq\" (UniqueName: \"kubernetes.io/projected/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-kube-api-access-8vqpq\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:57:11.161845 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.161828    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khb5q\" (UniqueName: \"kubernetes.io/projected/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-kube-api-access-khb5q\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.162074 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.162061    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5424\" (UniqueName: \"kubernetes.io/projected/6fe540a2-725b-4723-bb8c-11b83afe9b2a-kube-api-access-p5424\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.232389 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232312    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-netns\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.232389 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232348    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-log-socket\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.232389 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232372    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-sys\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232395    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04e386bc-2d2b-438e-bb10-af8b05ea5b30-hosts-file\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232415    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-netns\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232423    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-log-socket\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232431    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-bin\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232459    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-sys\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232467    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-bin\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232474    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/04e386bc-2d2b-438e-bb10-af8b05ea5b30-hosts-file\") pod \"node-resolver-lxg8q\" (UID: \"04e386bc-2d2b-438e-bb10-af8b05ea5b30\") " pod="openshift-dns/node-resolver-lxg8q"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232475    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-kubelet-config\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232500    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-device-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232528    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-kubelet-config\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232581    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-device-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.232617 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232618    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cnibin\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232644    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-netns\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232657    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-cnibin\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232673    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-socket-dir-parent\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232686    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-netns\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232699    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-ovn\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232729    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-socket-dir-parent\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232742    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232754    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-ovn\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232782    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-systemd-units\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232814    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232844    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-systemd\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232865    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-systemd-units\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232866    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-system-cni-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232884    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-node-log\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232889    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lfmxr\" (UID: \"d4947c1f-d461-4bd2-9cb0-4f7ff84b9ee3\") " pod="openshift-multus/multus-additional-cni-plugins-lfmxr"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232937    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-node-log\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232942    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-systemd\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232943    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-system-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.232986    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-k8s-cni-cncf-io\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233009    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-bin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233032    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a3053e-bb6f-4678-b865-4bb0e24f7dad-host\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233034    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-system-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233052    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-k8s-cni-cncf-io\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233059    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a3053e-bb6f-4678-b865-4bb0e24f7dad-host\") pod \"node-ca-vxvtw\" (UID: \"91a3053e-bb6f-4678-b865-4bb0e24f7dad\") " pod="openshift-image-registry/node-ca-vxvtw"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233071    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-bin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233094    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-kubelet\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233109    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-etc-kubernetes\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233125    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-kubelet\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233142    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cnibin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233156    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-multus\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233173    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysconfig\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233190    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-wtmp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233226    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-etc-kubernetes\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.233224 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233225    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-crio-socket\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233254    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysconfig\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233229    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-var-lib-cni-multus\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233268    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-sys-fs\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233263    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-cnibin\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233273    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-node-exporter-wtmp\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233284    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-run\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233311    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-systemd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233313    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-run\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233264    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ffa15d5-c851-4b6e-bfc7-faa42f332f0b-crio-socket\") pod \"insights-runtime-extractor-zwh9d\" (UID: \"0ffa15d5-c851-4b6e-bfc7-faa42f332f0b\") " pod="openshift-insights/insights-runtime-extractor-zwh9d"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233329    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233334    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-sys-fs\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233360    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233358    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-registration-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233387    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-lib-modules\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233406    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233364    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-systemd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233387    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-registration-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233426    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-etc-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233441    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233455    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-conf\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233460    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-cni-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233476    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-var-lib-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233485    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-etc-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233502    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233509    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-var-lib-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233502    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-lib-modules\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233520    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-sys\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233534    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-run-openvswitch\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233553    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-sys\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233556    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-run-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233599    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233559    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233622    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-conf\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233638    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-var-lib-kubelet\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233661    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-netd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233680    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-multus-certs\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233717    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-host-slash\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233744    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-var-lib-kubelet\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233748    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233771    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-cni-netd\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233775    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-root\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233722    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-host-run-multus-certs\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233802    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-kubelet\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233819    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eeef7c4-96d5-4b07-8e8d-d8dff6385427-root\") pod \"node-exporter-z7x2c\" (UID: \"2eeef7c4-96d5-4b07-8e8d-d8dff6385427\") " pod="openshift-monitoring/node-exporter-z7x2c"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233805    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f557c0f3-8e66-416c-8c4b-cebb3fd60a25-host-slash\") pod \"iptables-alerter-6s62t\" (UID: \"f557c0f3-8e66-416c-8c4b-cebb3fd60a25\") " pod="openshift-network-operator/iptables-alerter-6s62t"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233835    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-kubelet\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233832    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-socket-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233874    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-sysctl-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233877    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-dbus\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233887    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab-socket-dir\") pod \"aws-ebs-csi-driver-node-24fpk\" (UID: \"29bc3bca-4b3f-4e7d-a14b-89228a7fc8ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24fpk"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233932    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-slash\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233980    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-kubernetes\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233940    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08444539-c0d4-4b26-9bbd-c75c21b32a21-dbus\") pod \"global-pull-secret-syncer-trnpq\" (UID: \"08444539-c0d4-4b26-9bbd-c75c21b32a21\") " pod="kube-system/global-pull-secret-syncer-trnpq"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.233995    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-hostroot\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234030    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-kubernetes\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234031    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-conf-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234006    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-slash\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234037    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-hostroot\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234055    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234075    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-modprobe-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234087    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-host\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234104    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c3da85c-aef5-4f3f-9381-8665d944c858-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krkmg\" (UID: \"0c3da85c-aef5-4f3f-9381-8665d944c858\") " pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234142    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc9f7717-0e17-4ca2-aaa2-286a398e3435-multus-conf-dir\") pod \"multus-7zqrb\" (UID: \"dc9f7717-0e17-4ca2-aaa2-286a398e3435\") " pod="openshift-multus/multus-7zqrb"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234154    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-host\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.234339 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.234171    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6fe540a2-725b-4723-bb8c-11b83afe9b2a-etc-modprobe-d\") pod \"tuned-hmlkj\" (UID: \"6fe540a2-725b-4723-bb8c-11b83afe9b2a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmlkj"
Oct 30 17:57:11.296180 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.296164    5746 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 17:57:11.583048 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.582952    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.601988 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.601964    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:11.637953 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.637925    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.637972    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.638004    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.638026    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.638052    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638057    5746 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.638091    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638128    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:12.638110467 +0000 UTC m=+2.804928577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638140    5746 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 17:57:11.638174 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638148    5746 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:11.638643 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638179    5746 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 17:57:11.638643 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638152    5746 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:11.638643 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638190    5746 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:11.638643 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638196    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:12.638184846 +0000 UTC m=+2.805002951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"service-ca" not registered
Oct 30 17:57:11.638643 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638237    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:12.638223471 +0000 UTC m=+2.805041581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:11.638643 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638254    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:12.638244193 +0000 UTC m=+2.805062304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:11.638643 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638267    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:12.638259089 +0000 UTC m=+2.805077194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-config" not registered
Oct 30 17:57:11.638643 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.638277    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:12.638271657 +0000 UTC m=+2.805089762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:11.738524 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:11.738485    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:11.738761 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.738635    5746 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 17:57:11.738761 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.738658    5746 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 17:57:11.738761 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.738668    5746 projected.go:196] Error preparing data for projected volume kube-api-access-2z8q6 for pod openshift-console/console-64557c6d97-hgqx4: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:11.738761 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:11.738713    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6 podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:12.7387006 +0000 UTC m=+2.905518705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2z8q6" (UniqueName: "kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:12.312412 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:12.312381    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2ljs5"
Oct 30 17:57:12.647309 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:12.647274    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:12.647490 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:12.647329    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:12.647490 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:12.647369    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:12.647490 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:12.647413    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:12.647490 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647417    5746 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 17:57:12.647490 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:12.647453    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:12.647490 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:12.647474    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:12.647490 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647495    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:14.647475907 +0000 UTC m=+4.814294031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"service-ca" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647533    5746 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647536    5746 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647591    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:14.647578862 +0000 UTC m=+4.814396982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647606    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:14.647599129 +0000 UTC m=+4.814417237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647632    5746 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647644    5746 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647654    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:14.647648271 +0000 UTC m=+4.814466419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647683    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:14.647668099 +0000 UTC m=+4.814486211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-config" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647702    5746 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:12.647838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.647731    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:14.647721957 +0000 UTC m=+4.814540087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:12.748667 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:12.748633    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:12.748838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.748808    5746 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 17:57:12.748838 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.748832    5746 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 17:57:12.748947 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.748843    5746 projected.go:196] Error preparing data for projected volume kube-api-access-2z8q6 for pod openshift-console/console-64557c6d97-hgqx4: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:12.748947 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.748897    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6 podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:14.748880888 +0000 UTC m=+4.915698993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2z8q6" (UniqueName: "kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:12.793176 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:12.793143    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 17:57:14.099253 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.099215    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerStarted","Data":"3ed42d9c42a565d9edb3d73d8f7c74e0e2074d8e91c477413ee4df483bd786c3"}
Oct 30 17:57:14.099642 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.099284    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerStarted","Data":"24e32236a021bb3c3164dee108750625bc29a9d4ef910c95e0db25e763193e69"}
Oct 30 17:57:14.099642 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.099310    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerStarted","Data":"5aec4d4fc1c974894aff570401c4dc29a5e27faa3b29ad6f47b75cabb988b578"}
Oct 30 17:57:14.099642 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.099339    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerStarted","Data":"c91e02d18b59cc0397d82182087a4d70013dc5b57d3d3a13c0ac232824c37026"}
Oct 30 17:57:14.666370 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.666332    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:14.666370 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.666374    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:14.666586 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.666421    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:14.666586 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.666452    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:14.666586 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666475    5746 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:14.666586 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666530    5746 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 17:57:14.666586 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666534    5746 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:14.666586 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666544    5746 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 17:57:14.666586 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666563    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:18.666542294 +0000 UTC m=+8.833360418 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:14.666586 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.666483    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:14.666847 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666596    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:18.666580481 +0000 UTC m=+8.833398602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-config" not registered
Oct 30 17:57:14.666847 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666604    5746 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:14.666847 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666617    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:18.666607707 +0000 UTC m=+8.833425815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:14.666847 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666632    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:18.66662503 +0000 UTC m=+8.833443136 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"service-ca" not registered
Oct 30 17:57:14.666847 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666666    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:18.666651535 +0000 UTC m=+8.833469655 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:14.666847 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.666696    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:14.666847 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666781    5746 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:14.666847 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.666807    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:18.66679903 +0000 UTC m=+8.833617150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:14.768114 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:14.768082    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:14.768277 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.768233    5746 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 17:57:14.768277 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.768252    5746 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 17:57:14.768277 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.768263    5746 projected.go:196] Error preparing data for projected volume kube-api-access-2z8q6 for pod openshift-console/console-64557c6d97-hgqx4: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:14.768371 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:14.768309    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6 podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:18.768297712 +0000 UTC m=+8.935115844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2z8q6" (UniqueName: "kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:16.120250 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:16.120219    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:18.707321 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:18.707227    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:18.707321 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:18.707291    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:18.707333    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707355    5746 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:18.707368    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707391    5746 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:18.707414    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707429    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:26.707409175 +0000 UTC m=+16.874227286 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"service-ca" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707451    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:26.707444557 +0000 UTC m=+16.874262673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707468    5746 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707476    5746 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707492    5746 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707508    5746 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:18.707478    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707525    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:26.707508467 +0000 UTC m=+16.874326592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-config" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707557    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:26.707550142 +0000 UTC m=+16.874368248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707570    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:26.707565033 +0000 UTC m=+16.874383137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:18.707785 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.707581    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:26.707576006 +0000 UTC m=+16.874394110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:18.808806 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:18.808775    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:18.808980 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.808931    5746 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 17:57:18.808980 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.808946    5746 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 17:57:18.808980 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.808955    5746 projected.go:196] Error preparing data for projected volume kube-api-access-2z8q6 for pod openshift-console/console-64557c6d97-hgqx4: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:18.809110 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:18.809002    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6 podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:26.808989713 +0000 UTC m=+16.975807819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2z8q6" (UniqueName: "kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:18.833724 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:18.833700    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:57:20.246331 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:20.246301    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:20.246821 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:20.246501    5746 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 17:57:26.152040 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:26.151987    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s6687"
Oct 30 17:57:26.781815 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:26.781772    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:26.781835    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:26.781854    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:26.781886    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.781941    5746 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.781972    5746 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:26.781979    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782011    5746 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782012    5746 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782027    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:42.782008554 +0000 UTC m=+32.948826677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:26.782050    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:26.782057 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782056    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:42.782041642 +0000 UTC m=+32.948859779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 17:57:26.782505 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782077    5746 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:26.782505 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782082    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:42.782071418 +0000 UTC m=+32.948889523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"service-ca" not registered
Oct 30 17:57:26.782505 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782100    5746 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 17:57:26.782505 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782107    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:42.78209794 +0000 UTC m=+32.948916050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 17:57:26.782505 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782134    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:42.782125218 +0000 UTC m=+32.948943332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 17:57:26.782505 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.782149    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:42.782142495 +0000 UTC m=+32.948960605 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : object "openshift-console"/"console-config" not registered
Oct 30 17:57:26.883319 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:26.883280    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"console-64557c6d97-hgqx4\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") " pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:26.883482 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.883417    5746 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 17:57:26.883482 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.883431    5746 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 17:57:26.883482 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.883440    5746 projected.go:196] Error preparing data for projected volume kube-api-access-2z8q6 for pod openshift-console/console-64557c6d97-hgqx4: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:26.883604 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:26.883495    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6 podName:50c43e52-8525-45ad-9af8-026b029f538b nodeName:}" failed. No retries permitted until 2026-10-30 17:57:42.883482343 +0000 UTC m=+33.050300449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2z8q6" (UniqueName: "kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6") pod "console-64557c6d97-hgqx4" (UID: "50c43e52-8525-45ad-9af8-026b029f538b") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 17:57:28.536108 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:28.536075    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krkmg"
Oct 30 17:57:33.751950 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:33.751896    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:33.752408 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:33.752281    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="prometheus" containerID="cri-o://c63380af2646ba58fc4a6ba1c0edccd9dc5cabb3c2dda5e4496daa4d09bac399" gracePeriod=600
Oct 30 17:57:33.752408 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:33.752309    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy-web" containerID="cri-o://5aec4d4fc1c974894aff570401c4dc29a5e27faa3b29ad6f47b75cabb988b578" gracePeriod=600
Oct 30 17:57:33.752527 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:33.752370    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="config-reloader" containerID="cri-o://aa5f3e44c20d35aebaf7c71c801b093fe3fd4e880b818d766986f66acfb19ecb" gracePeriod=600
Oct 30 17:57:33.752527 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:33.752282    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3ed42d9c42a565d9edb3d73d8f7c74e0e2074d8e91c477413ee4df483bd786c3" gracePeriod=600
Oct 30 17:57:33.752527 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:33.752381    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy" containerID="cri-o://24e32236a021bb3c3164dee108750625bc29a9d4ef910c95e0db25e763193e69" gracePeriod=600
Oct 30 17:57:33.752665 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:33.752565    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="thanos-sidecar" containerID="cri-o://c91e02d18b59cc0397d82182087a4d70013dc5b57d3d3a13c0ac232824c37026" gracePeriod=600
Oct 30 17:57:34.172431 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172396    5746 generic.go:356] "Generic (PLEG): container finished" podID="7c9e9544-bb16-400e-b614-f06894ad09da" containerID="3ed42d9c42a565d9edb3d73d8f7c74e0e2074d8e91c477413ee4df483bd786c3" exitCode=0
Oct 30 17:57:34.172431 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172424    5746 generic.go:356] "Generic (PLEG): container finished" podID="7c9e9544-bb16-400e-b614-f06894ad09da" containerID="24e32236a021bb3c3164dee108750625bc29a9d4ef910c95e0db25e763193e69" exitCode=0
Oct 30 17:57:34.172431 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172432    5746 generic.go:356] "Generic (PLEG): container finished" podID="7c9e9544-bb16-400e-b614-f06894ad09da" containerID="5aec4d4fc1c974894aff570401c4dc29a5e27faa3b29ad6f47b75cabb988b578" exitCode=0
Oct 30 17:57:34.172431 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172439    5746 generic.go:356] "Generic (PLEG): container finished" podID="7c9e9544-bb16-400e-b614-f06894ad09da" containerID="c91e02d18b59cc0397d82182087a4d70013dc5b57d3d3a13c0ac232824c37026" exitCode=0
Oct 30 17:57:34.172690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172448    5746 generic.go:356] "Generic (PLEG): container finished" podID="7c9e9544-bb16-400e-b614-f06894ad09da" containerID="aa5f3e44c20d35aebaf7c71c801b093fe3fd4e880b818d766986f66acfb19ecb" exitCode=0
Oct 30 17:57:34.172690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172424    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerDied","Data":"3ed42d9c42a565d9edb3d73d8f7c74e0e2074d8e91c477413ee4df483bd786c3"}
Oct 30 17:57:34.172690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172515    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerDied","Data":"24e32236a021bb3c3164dee108750625bc29a9d4ef910c95e0db25e763193e69"}
Oct 30 17:57:34.172690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172543    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerDied","Data":"5aec4d4fc1c974894aff570401c4dc29a5e27faa3b29ad6f47b75cabb988b578"}
Oct 30 17:57:34.172690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172570    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerDied","Data":"c91e02d18b59cc0397d82182087a4d70013dc5b57d3d3a13c0ac232824c37026"}
Oct 30 17:57:34.172690 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:34.172593    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerDied","Data":"aa5f3e44c20d35aebaf7c71c801b093fe3fd4e880b818d766986f66acfb19ecb"}
Oct 30 17:57:36.012505 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.012459    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-64557c6d97-hgqx4" podUID="50c43e52-8525-45ad-9af8-026b029f538b" containerName="console" containerID="cri-o://d33c43a3bc463ca30bffcc883b8080288678359d574f7f30a60b3a5b229743bb" gracePeriod=15
Oct 30 17:57:36.181483 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.181450    5746 generic.go:356] "Generic (PLEG): container finished" podID="50c43e52-8525-45ad-9af8-026b029f538b" containerID="d33c43a3bc463ca30bffcc883b8080288678359d574f7f30a60b3a5b229743bb" exitCode=0
Oct 30 17:57:36.181619 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.181531    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64557c6d97-hgqx4" event={"ID":"50c43e52-8525-45ad-9af8-026b029f538b","Type":"ContainerDied","Data":"d33c43a3bc463ca30bffcc883b8080288678359d574f7f30a60b3a5b229743bb"}
Oct 30 17:57:36.244332 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.244308    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:36.374511 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.374471    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") pod \"50c43e52-8525-45ad-9af8-026b029f538b\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") "
Oct 30 17:57:36.374706 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.374529    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp\" (UniqueName: \"kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp\") pod \"50c43e52-8525-45ad-9af8-026b029f538b\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") "
Oct 30 17:57:36.374706 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.374557    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") pod \"50c43e52-8525-45ad-9af8-026b029f538b\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") "
Oct 30 17:57:36.374706 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.374578    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") pod \"50c43e52-8525-45ad-9af8-026b029f538b\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") "
Oct 30 17:57:36.374706 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.374609    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") pod \"50c43e52-8525-45ad-9af8-026b029f538b\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") "
Oct 30 17:57:36.374706 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.374684    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") pod \"50c43e52-8525-45ad-9af8-026b029f538b\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") "
Oct 30 17:57:36.374998 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.374739    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") pod \"50c43e52-8525-45ad-9af8-026b029f538b\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") "
Oct 30 17:57:36.374998 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.374773    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") pod \"50c43e52-8525-45ad-9af8-026b029f538b\" (UID: \"50c43e52-8525-45ad-9af8-026b029f538b\") "
Oct 30 17:57:36.375106 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.375015    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp" pod "50c43e52-8525-45ad-9af8-026b029f538b" (UID: "50c43e52-8525-45ad-9af8-026b029f538b"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 17:57:36.375286 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.375233    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle" pod "50c43e52-8525-45ad-9af8-026b029f538b" (UID: "50c43e52-8525-45ad-9af8-026b029f538b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:36.375286 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.375270    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert" pod "50c43e52-8525-45ad-9af8-026b029f538b" (UID: "50c43e52-8525-45ad-9af8-026b029f538b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:36.375396 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.375276    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config" pod "50c43e52-8525-45ad-9af8-026b029f538b" (UID: "50c43e52-8525-45ad-9af8-026b029f538b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:36.375396 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.375280    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca" pod "50c43e52-8525-45ad-9af8-026b029f538b" (UID: "50c43e52-8525-45ad-9af8-026b029f538b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:36.377350 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.377319    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert" pod "50c43e52-8525-45ad-9af8-026b029f538b" (UID: "50c43e52-8525-45ad-9af8-026b029f538b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:36.377482 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.377458    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6" pod "50c43e52-8525-45ad-9af8-026b029f538b" (UID: "50c43e52-8525-45ad-9af8-026b029f538b"). InnerVolumeSpecName "kube-api-access-2z8q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 17:57:36.377527 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.377469    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config" pod "50c43e52-8525-45ad-9af8-026b029f538b" (UID: "50c43e52-8525-45ad-9af8-026b029f538b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:36.475704 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.475659    5746 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-console-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:36.475704 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.475696    5746 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-oauth-serving-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:36.475704 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.475707    5746 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-service-ca\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:36.475704 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.475716    5746 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-oauth-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:36.476026 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.475724    5746 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50c43e52-8525-45ad-9af8-026b029f538b-tmp\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:36.476026 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.475733    5746 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50c43e52-8525-45ad-9af8-026b029f538b-trusted-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:36.476026 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.475742    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z8q6\" (UniqueName: \"kubernetes.io/projected/50c43e52-8525-45ad-9af8-026b029f538b-kube-api-access-2z8q6\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:36.476026 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.475750    5746 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50c43e52-8525-45ad-9af8-026b029f538b-console-serving-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:36.582997 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:36.582944    5746 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="c63380af2646ba58fc4a6ba1c0edccd9dc5cabb3c2dda5e4496daa4d09bac399" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 17:57:36.583095 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:36.583019    5746 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="c63380af2646ba58fc4a6ba1c0edccd9dc5cabb3c2dda5e4496daa4d09bac399" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 17:57:36.583095 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:36.583036    5746 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="c63380af2646ba58fc4a6ba1c0edccd9dc5cabb3c2dda5e4496daa4d09bac399" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 17:57:36.583095 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:36.583052    5746 prober.go:106] "Probe errored" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="prometheus" probeResult="unknown" err="rpc error: code = Canceled desc = context canceled"
Oct 30 17:57:37.185591 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:37.185561    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64557c6d97-hgqx4" event={"ID":"50c43e52-8525-45ad-9af8-026b029f538b","Type":"ContainerDied","Data":"f7156936dbc3eef054d59472932ea7ad7caa7d8e53a270f84781cef90767b5c7"}
Oct 30 17:57:37.185591 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:37.185585    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64557c6d97-hgqx4"
Oct 30 17:57:37.186068 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:37.185634    5746 scope.go:122] "RemoveContainer" containerID="d33c43a3bc463ca30bffcc883b8080288678359d574f7f30a60b3a5b229743bb"
Oct 30 17:57:37.240264 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:37.240241    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64557c6d97-hgqx4"]
Oct 30 17:57:37.254034 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:37.254010    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64557c6d97-hgqx4"]
Oct 30 17:57:38.007613 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:38.007578    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50c43e52-8525-45ad-9af8-026b029f538b" path="/var/lib/kubelet/pods/50c43e52-8525-45ad-9af8-026b029f538b/volumes"
Oct 30 17:57:38.835757 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:38.835731    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d9ml4"
Oct 30 17:57:41.203296 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.203266    5746 generic.go:356] "Generic (PLEG): container finished" podID="7c9e9544-bb16-400e-b614-f06894ad09da" containerID="c63380af2646ba58fc4a6ba1c0edccd9dc5cabb3c2dda5e4496daa4d09bac399" exitCode=0
Oct 30 17:57:41.203644 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.203301    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerDied","Data":"c63380af2646ba58fc4a6ba1c0edccd9dc5cabb3c2dda5e4496daa4d09bac399"}
Oct 30 17:57:41.292554 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.292531    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:41.410974 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.410884    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.410974 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.410939    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.410974 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.410959    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.410978    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.410997    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411033    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411405 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411266    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411405 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411350    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411515 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411405    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411515 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411442    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411515 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411496    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411662 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411508    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:41.411719 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411662    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:41.411774 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411743    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411886 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411780    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411886 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411846    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.411886 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411879    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.412172 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.411958    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.412172 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.412029    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.412172 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.412091    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.412172 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.412159    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.412381 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.412193    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1\") pod \"7c9e9544-bb16-400e-b614-f06894ad09da\" (UID: \"7c9e9544-bb16-400e-b614-f06894ad09da\") "
Oct 30 17:57:41.412516 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.412497    5746 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.412607 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.412522    5746 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-metrics-client-ca\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.412880 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.412857    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:41.413190 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.413167    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:41.414031 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.413987    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 17:57:41.414031 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.413998    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.415202 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.414266    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:41.415202 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.414785    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.415811 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.415772    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 17:57:41.416163 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.416056    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:41.416556 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.416501    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 17:57:41.416666 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.416533    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.417191 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.417069    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.417191 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.417156    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "kube-api-access-28mgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 17:57:41.418285 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.418259    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.418413 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.418377    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.418818 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.418793    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.419268 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.419247    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 17:57:41.419430 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.419411    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.428033 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.428007    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config" pod "7c9e9544-bb16-400e-b614-f06894ad09da" (UID: "7c9e9544-bb16-400e-b614-f06894ad09da"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 17:57:41.513116 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513084    5746 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-tls-assets\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513116 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513111    5746 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-metrics-client-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513116 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513121    5746 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-kube-rbac-proxy\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513130    5746 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-grpc-tls\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513142    5746 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513151    5746 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-web-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513159    5746 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513167    5746 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-db\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513176    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28mgj\" (UniqueName: \"kubernetes.io/projected/7c9e9544-bb16-400e-b614-f06894ad09da-kube-api-access-28mgj\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513185    5746 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513194    5746 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513203    5746 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513212    5746 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513220    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513228    5746 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513236    5746 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c9e9544-bb16-400e-b614-f06894ad09da-config-out\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513244    5746 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c9e9544-bb16-400e-b614-f06894ad09da-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:41.513337 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:41.513252    5746 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c9e9544-bb16-400e-b614-f06894ad09da-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 17:57:42.209957 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.209924    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c9e9544-bb16-400e-b614-f06894ad09da","Type":"ContainerDied","Data":"222e997a856f88fd98b37103b71fd17597db420f212199c6b2f7a93ce9b1a56d"}
Oct 30 17:57:42.210347 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.209980    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.210347 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.209998    5746 scope.go:122] "RemoveContainer" containerID="3ed42d9c42a565d9edb3d73d8f7c74e0e2074d8e91c477413ee4df483bd786c3"
Oct 30 17:57:42.217310 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.217290    5746 scope.go:122] "RemoveContainer" containerID="24e32236a021bb3c3164dee108750625bc29a9d4ef910c95e0db25e763193e69"
Oct 30 17:57:42.223974 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.223959    5746 scope.go:122] "RemoveContainer" containerID="5aec4d4fc1c974894aff570401c4dc29a5e27faa3b29ad6f47b75cabb988b578"
Oct 30 17:57:42.230591 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.230577    5746 scope.go:122] "RemoveContainer" containerID="c91e02d18b59cc0397d82182087a4d70013dc5b57d3d3a13c0ac232824c37026"
Oct 30 17:57:42.236714 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.236698    5746 scope.go:122] "RemoveContainer" containerID="aa5f3e44c20d35aebaf7c71c801b093fe3fd4e880b818d766986f66acfb19ecb"
Oct 30 17:57:42.245136 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.245097    5746 scope.go:122] "RemoveContainer" containerID="c63380af2646ba58fc4a6ba1c0edccd9dc5cabb3c2dda5e4496daa4d09bac399"
Oct 30 17:57:42.252139 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.252111    5746 scope.go:122] "RemoveContainer" containerID="b75756a6b05f4a5b01290d9a5a86376e4d82c40388d9b932e64cf8029d3fd207"
Oct 30 17:57:42.283789 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.283735    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:42.297534 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.297512    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:42.323578 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323554    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:42.323790 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323778    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="config-reloader"
Oct 30 17:57:42.323832 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323791    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="config-reloader"
Oct 30 17:57:42.323832 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323800    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="init-config-reloader"
Oct 30 17:57:42.323832 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323805    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="init-config-reloader"
Oct 30 17:57:42.323832 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323818    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="thanos-sidecar"
Oct 30 17:57:42.323832 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323822    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="thanos-sidecar"
Oct 30 17:57:42.323832 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323829    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy-web"
Oct 30 17:57:42.323832 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323833    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy-web"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323845    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323849    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323855    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy-thanos"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323859    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy-thanos"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323868    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="prometheus"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323872    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="prometheus"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323877    5746 cpu_manager.go:414] "containerMap: removing container" podUID="50c43e52-8525-45ad-9af8-026b029f538b" containerName="console"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323881    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50c43e52-8525-45ad-9af8-026b029f538b" containerName="console"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323950    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="config-reloader"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323959    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="thanos-sidecar"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323964    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy-web"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323970    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323974    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="kube-rbac-proxy-thanos"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323980    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" containerName="prometheus"
Oct 30 17:57:42.324047 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.323984    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="50c43e52-8525-45ad-9af8-026b029f538b" containerName="console"
Oct 30 17:57:42.327076 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.327060    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.344039 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344020    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 17:57:42.344208 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344069    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 17:57:42.344208 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344102    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 17:57:42.344533 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344219    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 17:57:42.344533 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344252    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 17:57:42.344533 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344349    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 17:57:42.344533 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344399    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 17:57:42.344937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344585    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 17:57:42.344937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344599    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 17:57:42.344937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344635    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hqs45\""
Oct 30 17:57:42.344937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344634    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 17:57:42.344937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344650    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-69ncnlhdiul6u\""
Oct 30 17:57:42.344937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344769    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 17:57:42.344937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344844    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 17:57:42.344937 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.344872    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 17:57:42.359153 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.359132    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 17:57:42.359273 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.359249    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 17:57:42.359555 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.359536    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:42.418794 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.418767    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.418925 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.418797    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.418925 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.418828    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.418925 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.418847    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/acd7341b-f7d5-4198-a608-37bf693bbc5b-config-out\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.418925 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.418900    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjjnw\" (UniqueName: \"kubernetes.io/projected/acd7341b-f7d5-4198-a608-37bf693bbc5b-kube-api-access-fjjnw\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419078 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.418943    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419078 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419018    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419078 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419035    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419078 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419050    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419078 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419069    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/acd7341b-f7d5-4198-a608-37bf693bbc5b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419241 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419085    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419241 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419179    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-config\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419241 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419203    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419241 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419220    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419241 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419236    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419405 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419270    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419405 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419314    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419405 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419352    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-web-config\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419405 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419370    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.419405 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.419395    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520073 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520042    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520077    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/acd7341b-f7d5-4198-a608-37bf693bbc5b-config-out\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520101    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjjnw\" (UniqueName: \"kubernetes.io/projected/acd7341b-f7d5-4198-a608-37bf693bbc5b-kube-api-access-fjjnw\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520120    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520149    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520184    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520199    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520223    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/acd7341b-f7d5-4198-a608-37bf693bbc5b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520245 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520249    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520282    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-config\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520470    5746 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520501    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520533    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520560    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520595    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520620    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520652    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-web-config\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520672    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.520685 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520683    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.521237 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520732    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.521237 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520774    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.521237 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520800    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.521237 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.520876    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.522238 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.521506    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.522238 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.521785    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.523256 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.522461    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.523598 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.523580    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/acd7341b-f7d5-4198-a608-37bf693bbc5b-config-out\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.523738 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.523651    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.523872 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.523731    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.524093 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.524071    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.524248 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.524226    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.525022 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.524997    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.525253 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.525231    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/acd7341b-f7d5-4198-a608-37bf693bbc5b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.525795 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.525772    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.526363 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.526339    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.526460 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.526419    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-web-config\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.526529 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.526510    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-config\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.526962 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.526944    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.528107 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.528088    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/acd7341b-f7d5-4198-a608-37bf693bbc5b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.528189 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.528169    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/acd7341b-f7d5-4198-a608-37bf693bbc5b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.545328 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.545279    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjjnw\" (UniqueName: \"kubernetes.io/projected/acd7341b-f7d5-4198-a608-37bf693bbc5b-kube-api-access-fjjnw\") pod \"prometheus-k8s-0\" (UID: \"acd7341b-f7d5-4198-a608-37bf693bbc5b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.644324 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.644297    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:42.790002 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:57:42.789964    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 17:57:42.981802 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:42.981772    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 17:57:42.985280 ip-10-0-131-31 kubenswrapper[5746]: W1030 17:57:42.985249    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd7341b_f7d5_4198_a608_37bf693bbc5b.slice/crio-d94bbe3f4d2df7955a07901a78be3e3707c6d1d125130c3fcb5f52d979950f15 WatchSource:0}: Error finding container d94bbe3f4d2df7955a07901a78be3e3707c6d1d125130c3fcb5f52d979950f15: Status 404 returned error can't find the container with id d94bbe3f4d2df7955a07901a78be3e3707c6d1d125130c3fcb5f52d979950f15
Oct 30 17:57:43.215509 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:43.215477    5746 generic.go:356] "Generic (PLEG): container finished" podID="acd7341b-f7d5-4198-a608-37bf693bbc5b" containerID="d01ba89df002bc7ad28f97d60b6f79e29c5e62fcf78d217641f6e7a9502975d8" exitCode=0
Oct 30 17:57:43.215870 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:43.215530    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"acd7341b-f7d5-4198-a608-37bf693bbc5b","Type":"ContainerDied","Data":"d01ba89df002bc7ad28f97d60b6f79e29c5e62fcf78d217641f6e7a9502975d8"}
Oct 30 17:57:43.215870 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:43.215590    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"acd7341b-f7d5-4198-a608-37bf693bbc5b","Type":"ContainerStarted","Data":"d94bbe3f4d2df7955a07901a78be3e3707c6d1d125130c3fcb5f52d979950f15"}
Oct 30 17:57:44.007834 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:44.007784    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c9e9544-bb16-400e-b614-f06894ad09da" path="/var/lib/kubelet/pods/7c9e9544-bb16-400e-b614-f06894ad09da/volumes"
Oct 30 17:57:44.221920 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:44.221865    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"acd7341b-f7d5-4198-a608-37bf693bbc5b","Type":"ContainerStarted","Data":"330df633c4be71caa256466d6a57b785fc8eaafaa999051bed2af061e1154608"}
Oct 30 17:57:44.222317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:44.221973    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"acd7341b-f7d5-4198-a608-37bf693bbc5b","Type":"ContainerStarted","Data":"f74fb5c88dd3d86d9ef8655110e4ca2d460c4e22194de55bdb1a164b3609c839"}
Oct 30 17:57:44.222317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:44.222016    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"acd7341b-f7d5-4198-a608-37bf693bbc5b","Type":"ContainerStarted","Data":"35c12d5b8e36b99540153fc9da13d99ef926084b5341757cc9a93b6389a55954"}
Oct 30 17:57:44.222317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:44.222054    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"acd7341b-f7d5-4198-a608-37bf693bbc5b","Type":"ContainerStarted","Data":"73c27a7ff67e260dc50124631543ed38c74b36e138f5ff90b7e2268f34a34ca0"}
Oct 30 17:57:44.222317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:44.222089    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"acd7341b-f7d5-4198-a608-37bf693bbc5b","Type":"ContainerStarted","Data":"d2fc652de115a8b94f57c8dac3cd6094ff81ac5a56f39f22c07b47abe7b76fe5"}
Oct 30 17:57:44.222317 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:44.222124    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"acd7341b-f7d5-4198-a608-37bf693bbc5b","Type":"ContainerStarted","Data":"bacd270003a2c4bff674b1398bc2e7035df0d8b7ddfb57cbe15f2a66d89a7187"}
Oct 30 17:57:44.296051 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:44.295956    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.280965832 podStartE2EDuration="2.295943265s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.977433ms" isStatefulPod=false podCreationTimestamp="2026-10-30 17:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 17:57:44.294976868 +0000 UTC m=+34.461795013" watchObservedRunningTime="2026-10-30 17:57:44.295943265 +0000 UTC m=+34.462761389"
Oct 30 17:57:47.645355 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:47.645315    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:48.649462 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:48.649414    5746 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:48.665284 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:48.665263    5746 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:57:49.254156 ip-10-0-131-31 kubenswrapper[5746]: I1030 17:57:49.254133    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 17:58:12.789371 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:58:12.789336    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 17:58:42.788977 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:58:42.788939    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 17:59:12.789488 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:59:12.789454    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 17:59:42.789696 ip-10-0-131-31 kubenswrapper[5746]: E1030 17:59:42.789660    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:00:12.792245 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:00:12.792209    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:00:42.790260 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:00:42.790180    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:01:12.789107 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:01:12.789068    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:01:42.789304 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:01:42.789264    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:02:12.789466 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:02:12.789431    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:02:42.789431 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:02:42.789391    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:03:12.788708 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:03:12.788674    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:03:42.788998 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:03:42.788896    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:04:12.790685 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:04:12.790649    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:04:42.790022 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:04:42.789984    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:05:12.790323 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:05:12.790283    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:05:42.789929 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:05:42.789865    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:06:12.788758 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:06:12.788715    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:06:42.789827 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:06:42.789736    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:06:49.010104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.010070    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"]
Oct 30 18:06:49.013163 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.013141    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"
Oct 30 18:06:49.039997 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.039976    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 18:06:49.040116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.040017    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fzvm7\""
Oct 30 18:06:49.040116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.040054    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 18:06:49.040873 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.040849    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"]
Oct 30 18:06:49.085644 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.085607    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swz2s\" (UniqueName: \"kubernetes.io/projected/79b58365-fb8b-4b06-bc75-67b730c29233-kube-api-access-swz2s\") pod \"cert-manager-cainjector-5c57bbd4fc-jw9xx\" (UID: \"79b58365-fb8b-4b06-bc75-67b730c29233\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"
Oct 30 18:06:49.085644 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.085645    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79b58365-fb8b-4b06-bc75-67b730c29233-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-jw9xx\" (UID: \"79b58365-fb8b-4b06-bc75-67b730c29233\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"
Oct 30 18:06:49.187002 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.186960    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swz2s\" (UniqueName: \"kubernetes.io/projected/79b58365-fb8b-4b06-bc75-67b730c29233-kube-api-access-swz2s\") pod \"cert-manager-cainjector-5c57bbd4fc-jw9xx\" (UID: \"79b58365-fb8b-4b06-bc75-67b730c29233\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"
Oct 30 18:06:49.187198 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.187012    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79b58365-fb8b-4b06-bc75-67b730c29233-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-jw9xx\" (UID: \"79b58365-fb8b-4b06-bc75-67b730c29233\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"
Oct 30 18:06:49.210701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.210668    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79b58365-fb8b-4b06-bc75-67b730c29233-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-jw9xx\" (UID: \"79b58365-fb8b-4b06-bc75-67b730c29233\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"
Oct 30 18:06:49.210963 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.210941    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swz2s\" (UniqueName: \"kubernetes.io/projected/79b58365-fb8b-4b06-bc75-67b730c29233-kube-api-access-swz2s\") pod \"cert-manager-cainjector-5c57bbd4fc-jw9xx\" (UID: \"79b58365-fb8b-4b06-bc75-67b730c29233\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"
Oct 30 18:06:49.342244 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.342151    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"
Oct 30 18:06:49.640784 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.640745    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx"]
Oct 30 18:06:49.645594 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:06:49.645567    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79b58365_fb8b_4b06_bc75_67b730c29233.slice/crio-1b22e07e232be43384c460c82056d07b00412cd5dfc5c580128dd9512b8a7b16 WatchSource:0}: Error finding container 1b22e07e232be43384c460c82056d07b00412cd5dfc5c580128dd9512b8a7b16: Status 404 returned error can't find the container with id 1b22e07e232be43384c460c82056d07b00412cd5dfc5c580128dd9512b8a7b16
Oct 30 18:06:49.647374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.647358    5746 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:06:49.912931 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:49.912833    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx" event={"ID":"79b58365-fb8b-4b06-bc75-67b730c29233","Type":"ContainerStarted","Data":"1b22e07e232be43384c460c82056d07b00412cd5dfc5c580128dd9512b8a7b16"}
Oct 30 18:06:52.925887 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:52.925853    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx" event={"ID":"79b58365-fb8b-4b06-bc75-67b730c29233","Type":"ContainerStarted","Data":"4a0f446be6b190980be092e7d228d61a821e45db197a101aa106dc5c66abd51d"}
Oct 30 18:06:52.966340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:06:52.966286    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-jw9xx" podStartSLOduration=2.168937129 podStartE2EDuration="4.966269493s" totalImagesPullingTime="2.797332364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:06:52.965638141 +0000 UTC m=+583.132456269" watchObservedRunningTime="2026-10-30 18:06:52.966269493 +0000 UTC m=+583.133087623"
Oct 30 18:07:12.789768 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:07:12.789727    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:07:42.790414 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:07:42.790377    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:08:12.789950 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:08:12.789899    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:08:42.789335 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:08:42.789294    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:09:01.775848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.775812    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"]
Oct 30 18:09:01.779116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.779094    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:01.800963 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.800944    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-opentelemetry-operator\"/\"kube-root-ca.crt\""
Oct 30 18:09:01.803242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.803224    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-opentelemetry-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:09:01.812850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.812831    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"]
Oct 30 18:09:01.892285 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.892261    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-opentelemetry-operator\"/\"opentelemetry-operator-webhook-service-cert\""
Oct 30 18:09:01.892441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.892425    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-opentelemetry-operator\"/\"opentelemetry-operator-controller-manager-dockercfg-85bvp\""
Oct 30 18:09:01.944998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.944971    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43fe8eea-df09-435c-a2b9-b3dab5111659-webhook-cert\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:01.945104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.945006    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74tzb\" (UniqueName: \"kubernetes.io/projected/43fe8eea-df09-435c-a2b9-b3dab5111659-kube-api-access-74tzb\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:01.945104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:01.945034    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43fe8eea-df09-435c-a2b9-b3dab5111659-apiservice-cert\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:02.045880 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:02.045797    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43fe8eea-df09-435c-a2b9-b3dab5111659-webhook-cert\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:02.045880 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:02.045840    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74tzb\" (UniqueName: \"kubernetes.io/projected/43fe8eea-df09-435c-a2b9-b3dab5111659-kube-api-access-74tzb\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:02.045880 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:02.045882    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43fe8eea-df09-435c-a2b9-b3dab5111659-apiservice-cert\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:02.048424 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:02.048395    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43fe8eea-df09-435c-a2b9-b3dab5111659-apiservice-cert\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:02.048533 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:02.048421    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43fe8eea-df09-435c-a2b9-b3dab5111659-webhook-cert\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:02.070312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:02.070283    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74tzb\" (UniqueName: \"kubernetes.io/projected/43fe8eea-df09-435c-a2b9-b3dab5111659-kube-api-access-74tzb\") pod \"opentelemetry-operator-webhook-79fb9f5d66-6dj6m\" (UID: \"43fe8eea-df09-435c-a2b9-b3dab5111659\") " pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:02.089722 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:02.089691    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:02.415504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:02.415472    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"]
Oct 30 18:09:02.419181 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:09:02.419151    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fe8eea_df09_435c_a2b9_b3dab5111659.slice/crio-2958bc9801d61a5d6ff5cd326ca3b56f898ed9c607be173d653a92532a366479 WatchSource:0}: Error finding container 2958bc9801d61a5d6ff5cd326ca3b56f898ed9c607be173d653a92532a366479: Status 404 returned error can't find the container with id 2958bc9801d61a5d6ff5cd326ca3b56f898ed9c607be173d653a92532a366479
Oct 30 18:09:03.342895 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:03.342858    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m" event={"ID":"43fe8eea-df09-435c-a2b9-b3dab5111659","Type":"ContainerStarted","Data":"2958bc9801d61a5d6ff5cd326ca3b56f898ed9c607be173d653a92532a366479"}
Oct 30 18:09:05.352972 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:05.352944    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m" event={"ID":"43fe8eea-df09-435c-a2b9-b3dab5111659","Type":"ContainerStarted","Data":"02f13d823d0e606ea28f0abefdc2bfff61b0c576681320ac2e8fa1223c6df8e0"}
Oct 30 18:09:05.353393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:05.353058    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:05.420671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:05.420621    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m" podStartSLOduration=1.816824845 podStartE2EDuration="4.420609468s" totalImagesPullingTime="2.603784623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:09:05.420173736 +0000 UTC m=+715.586991882" watchObservedRunningTime="2026-10-30 18:09:05.420609468 +0000 UTC m=+715.587427593"
Oct 30 18:09:12.789619 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:09:12.789578    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:09:13.734513 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.734474    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"]
Oct 30 18:09:13.739433 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.739413    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:13.758269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.758237    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"]
Oct 30 18:09:13.769456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.769428    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-observability-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:09:13.769580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.769428    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-observability-operator\"/\"observability-operator-sa-dockercfg-q75x9\""
Oct 30 18:09:13.769580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.769467    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-observability-operator\"/\"kube-root-ca.crt\""
Oct 30 18:09:13.769688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.769674    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-observability-operator\"/\"observability-operator-tls\""
Oct 30 18:09:13.836947 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.836900    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"observability-operator-tls\" (UniqueName: \"kubernetes.io/secret/75102a6f-3ea8-4ebb-ac0c-1de370b34c58-observability-operator-tls\") pod \"observability-operator-599c6f89d-c75ht\" (UID: \"75102a6f-3ea8-4ebb-ac0c-1de370b34c58\") " pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:13.837339 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.837013    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkzpg\" (UniqueName: \"kubernetes.io/projected/75102a6f-3ea8-4ebb-ac0c-1de370b34c58-kube-api-access-kkzpg\") pod \"observability-operator-599c6f89d-c75ht\" (UID: \"75102a6f-3ea8-4ebb-ac0c-1de370b34c58\") " pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:13.938152 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.938113    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"observability-operator-tls\" (UniqueName: \"kubernetes.io/secret/75102a6f-3ea8-4ebb-ac0c-1de370b34c58-observability-operator-tls\") pod \"observability-operator-599c6f89d-c75ht\" (UID: \"75102a6f-3ea8-4ebb-ac0c-1de370b34c58\") " pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:13.938339 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.938159    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkzpg\" (UniqueName: \"kubernetes.io/projected/75102a6f-3ea8-4ebb-ac0c-1de370b34c58-kube-api-access-kkzpg\") pod \"observability-operator-599c6f89d-c75ht\" (UID: \"75102a6f-3ea8-4ebb-ac0c-1de370b34c58\") " pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:13.941019 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.940993    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"observability-operator-tls\" (UniqueName: \"kubernetes.io/secret/75102a6f-3ea8-4ebb-ac0c-1de370b34c58-observability-operator-tls\") pod \"observability-operator-599c6f89d-c75ht\" (UID: \"75102a6f-3ea8-4ebb-ac0c-1de370b34c58\") " pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:13.961497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:13.961476    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkzpg\" (UniqueName: \"kubernetes.io/projected/75102a6f-3ea8-4ebb-ac0c-1de370b34c58-kube-api-access-kkzpg\") pod \"observability-operator-599c6f89d-c75ht\" (UID: \"75102a6f-3ea8-4ebb-ac0c-1de370b34c58\") " pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:14.051663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:14.051585    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:14.377192 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:14.377157    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"]
Oct 30 18:09:14.380417 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:09:14.380390    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75102a6f_3ea8_4ebb_ac0c_1de370b34c58.slice/crio-c47980bea3af1cde8a63db7cf38596add1adf308dabf59d4229a32e1d8453f21 WatchSource:0}: Error finding container c47980bea3af1cde8a63db7cf38596add1adf308dabf59d4229a32e1d8453f21: Status 404 returned error can't find the container with id c47980bea3af1cde8a63db7cf38596add1adf308dabf59d4229a32e1d8453f21
Oct 30 18:09:15.392144 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:15.392073    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht" event={"ID":"75102a6f-3ea8-4ebb-ac0c-1de370b34c58","Type":"ContainerStarted","Data":"c47980bea3af1cde8a63db7cf38596add1adf308dabf59d4229a32e1d8453f21"}
Oct 30 18:09:16.362358 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:16.362320    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-opentelemetry-operator/opentelemetry-operator-webhook-79fb9f5d66-6dj6m"
Oct 30 18:09:18.404378 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:18.404333    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht" event={"ID":"75102a6f-3ea8-4ebb-ac0c-1de370b34c58","Type":"ContainerStarted","Data":"2ca700c3687051034f5a8ecfc52ffd0d49c88811130fe393fa3acc0108fe7b8e"}
Oct 30 18:09:18.404873 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:18.404578    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:18.406222 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:18.406200    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht"
Oct 30 18:09:18.529527 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:18.529479    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-observability-operator/observability-operator-599c6f89d-c75ht" podStartSLOduration=2.294544574 podStartE2EDuration="5.529463963s" totalImagesPullingTime="3.234919389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:09:18.517326138 +0000 UTC m=+728.684144285" watchObservedRunningTime="2026-10-30 18:09:18.529463963 +0000 UTC m=+728.696282090"
Oct 30 18:09:42.789459 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:09:42.789420    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:09:54.889065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.889028    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"]
Oct 30 18:09:54.891044 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.891023    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:54.907813 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.907784    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators-redhat\"/\"loki-operator-metrics\""
Oct 30 18:09:54.907939 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.907873    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators-redhat\"/\"loki-operator-controller-manager-service-cert\""
Oct 30 18:09:54.910849 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.910834    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators-redhat\"/\"loki-operator-controller-manager-dockercfg-zcb2w\""
Oct 30 18:09:54.911215 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.911194    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators-redhat\"/\"openshift-service-ca.crt\""
Oct 30 18:09:54.911215 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.911194    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators-redhat\"/\"loki-operator-manager-config\""
Oct 30 18:09:54.911361 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.911243    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators-redhat\"/\"kube-root-ca.crt\""
Oct 30 18:09:54.920636 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.920609    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"]
Oct 30 18:09:54.923191 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.923175    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs8m7\" (UniqueName: \"kubernetes.io/projected/789a8f75-732a-48f1-bd0a-e7770b11ebb9-kube-api-access-hs8m7\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:54.923264 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.923203    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-webhook-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:54.923264 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.923224    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/789a8f75-732a-48f1-bd0a-e7770b11ebb9-manager-config\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:54.923353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.923334    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-apiservice-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:54.923454 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:54.923439    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"loki-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-loki-operator-metrics-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.024035 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.024001    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"loki-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-loki-operator-metrics-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.024035 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.024038    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8m7\" (UniqueName: \"kubernetes.io/projected/789a8f75-732a-48f1-bd0a-e7770b11ebb9-kube-api-access-hs8m7\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.024263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.024066    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-webhook-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.024263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.024176    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/789a8f75-732a-48f1-bd0a-e7770b11ebb9-manager-config\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.024263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.024235    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-apiservice-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.024878 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.024851    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/789a8f75-732a-48f1-bd0a-e7770b11ebb9-manager-config\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.026720 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.026699    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-webhook-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.026815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.026741    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-apiservice-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.026815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.026739    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"loki-operator-metrics-cert\" (UniqueName: \"kubernetes.io/secret/789a8f75-732a-48f1-bd0a-e7770b11ebb9-loki-operator-metrics-cert\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.049458 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.049433    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs8m7\" (UniqueName: \"kubernetes.io/projected/789a8f75-732a-48f1-bd0a-e7770b11ebb9-kube-api-access-hs8m7\") pod \"loki-operator-controller-manager-575f57df95-bljm6\" (UID: \"789a8f75-732a-48f1-bd0a-e7770b11ebb9\") " pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.201221 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.201143    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:55.522376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:55.522296    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"]
Oct 30 18:09:55.525698 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:09:55.525667    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod789a8f75_732a_48f1_bd0a_e7770b11ebb9.slice/crio-368be3b31ab9c5a57981ba6f9859f2d9cf7358497ecc908f024dee6587ba82a5 WatchSource:0}: Error finding container 368be3b31ab9c5a57981ba6f9859f2d9cf7358497ecc908f024dee6587ba82a5: Status 404 returned error can't find the container with id 368be3b31ab9c5a57981ba6f9859f2d9cf7358497ecc908f024dee6587ba82a5
Oct 30 18:09:56.533277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:56.533244    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6" event={"ID":"789a8f75-732a-48f1-bd0a-e7770b11ebb9","Type":"ContainerStarted","Data":"368be3b31ab9c5a57981ba6f9859f2d9cf7358497ecc908f024dee6587ba82a5"}
Oct 30 18:09:59.548215 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:59.548177    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6" event={"ID":"789a8f75-732a-48f1-bd0a-e7770b11ebb9","Type":"ContainerStarted","Data":"864e5b7010d1a6a68aeff65b9a63145450304df56427ba87467522ea0341c486"}
Oct 30 18:09:59.548678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:59.548236    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:09:59.615580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:09:59.615524    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6" podStartSLOduration=2.639430721 podStartE2EDuration="5.615511375s" totalImagesPullingTime="2.976080654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:09:59.603433439 +0000 UTC m=+769.770251576" watchObservedRunningTime="2026-10-30 18:09:59.615511375 +0000 UTC m=+769.782329502"
Oct 30 18:10:10.553618 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:10:10.553576    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators-redhat/loki-operator-controller-manager-575f57df95-bljm6"
Oct 30 18:10:12.790563 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:10:12.790523    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:10:42.789865 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:10:42.789827    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:11:02.438665 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.438633    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:11:02.440977 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.440961    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.455481 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.455458    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:11:02.456863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.456847    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:11:02.456938 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.456847    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:11:02.467373 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.467347    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:11:02.468078 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.468052    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:11:02.627987 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.627951    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n55pk\" (UniqueName: \"kubernetes.io/projected/45a348d0-e1f2-4966-a14f-5034036bc7c2-kube-api-access-n55pk\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.628209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.628023    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/45a348d0-e1f2-4966-a14f-5034036bc7c2-plugins\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.628209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.628117    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45a348d0-e1f2-4966-a14f-5034036bc7c2-config\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.729287 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.729192    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/45a348d0-e1f2-4966-a14f-5034036bc7c2-plugins\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.729287 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.729235    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45a348d0-e1f2-4966-a14f-5034036bc7c2-config\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.729287 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.729274    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n55pk\" (UniqueName: \"kubernetes.io/projected/45a348d0-e1f2-4966-a14f-5034036bc7c2-kube-api-access-n55pk\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.729698 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.729674    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/45a348d0-e1f2-4966-a14f-5034036bc7c2-plugins\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.729967 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.729948    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45a348d0-e1f2-4966-a14f-5034036bc7c2-config\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:02.752671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:02.752637    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n55pk\" (UniqueName: \"kubernetes.io/projected/45a348d0-e1f2-4966-a14f-5034036bc7c2-kube-api-access-n55pk\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.155378 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.155345    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.184026 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.183988    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.255813 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.255777    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.256055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.256028    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.270201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.270171    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.458082 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.458051    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.482839 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.482805    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.652070 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.652014    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:06.945677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:06.945648    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:11:06.948824 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:11:06.948800    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45a348d0_e1f2_4966_a14f_5034036bc7c2.slice/crio-748520b5b5fd5a2b01c8d5072d30a846a7322abb40851367b67a57e734207a5e WatchSource:0}: Error finding container 748520b5b5fd5a2b01c8d5072d30a846a7322abb40851367b67a57e734207a5e: Status 404 returned error can't find the container with id 748520b5b5fd5a2b01c8d5072d30a846a7322abb40851367b67a57e734207a5e
Oct 30 18:11:07.797423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:07.797384    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"45a348d0-e1f2-4966-a14f-5034036bc7c2","Type":"ContainerStarted","Data":"748520b5b5fd5a2b01c8d5072d30a846a7322abb40851367b67a57e734207a5e"}
Oct 30 18:11:10.812976 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:10.812936    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"45a348d0-e1f2-4966-a14f-5034036bc7c2","Type":"ContainerStarted","Data":"b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8"}
Oct 30 18:11:10.813437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:10.813021    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:10.854359 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:10.854307    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=8.211933167 podStartE2EDuration="11.854289363s" totalImagesPullingTime="3.642356196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:11:10.853665233 +0000 UTC m=+841.020483359" watchObservedRunningTime="2026-10-30 18:11:10.854289363 +0000 UTC m=+841.021107493"
Oct 30 18:11:12.790405 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:11:12.790367    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:11:21.820184 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:21.820150    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:35.901059 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:35.901026    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:11:35.901554 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:35.901244    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="45a348d0-e1f2-4966-a14f-5034036bc7c2" containerName="perses" containerID="cri-o://b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8" gracePeriod=30
Oct 30 18:11:36.142199 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.142175    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:36.199812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.199739    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45a348d0-e1f2-4966-a14f-5034036bc7c2-kube-api-access-n55pk\" (UniqueName: \"kubernetes.io/projected/45a348d0-e1f2-4966-a14f-5034036bc7c2-kube-api-access-n55pk\") pod \"45a348d0-e1f2-4966-a14f-5034036bc7c2\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") "
Oct 30 18:11:36.199975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.199865    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"45a348d0-e1f2-4966-a14f-5034036bc7c2\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") "
Oct 30 18:11:36.200028 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.199986    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45a348d0-e1f2-4966-a14f-5034036bc7c2-config\" (UniqueName: \"kubernetes.io/configmap/45a348d0-e1f2-4966-a14f-5034036bc7c2-config\") pod \"45a348d0-e1f2-4966-a14f-5034036bc7c2\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") "
Oct 30 18:11:36.200073 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.200042    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45a348d0-e1f2-4966-a14f-5034036bc7c2-plugins\" (UniqueName: \"kubernetes.io/empty-dir/45a348d0-e1f2-4966-a14f-5034036bc7c2-plugins\") pod \"45a348d0-e1f2-4966-a14f-5034036bc7c2\" (UID: \"45a348d0-e1f2-4966-a14f-5034036bc7c2\") "
Oct 30 18:11:36.200492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.200460    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a348d0-e1f2-4966-a14f-5034036bc7c2-config" pod "45a348d0-e1f2-4966-a14f-5034036bc7c2" (UID: "45a348d0-e1f2-4966-a14f-5034036bc7c2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:11:36.202258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.202227    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a348d0-e1f2-4966-a14f-5034036bc7c2-kube-api-access-n55pk" pod "45a348d0-e1f2-4966-a14f-5034036bc7c2" (UID: "45a348d0-e1f2-4966-a14f-5034036bc7c2"). InnerVolumeSpecName "kube-api-access-n55pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:11:36.204818 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.204795    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "45a348d0-e1f2-4966-a14f-5034036bc7c2" (UID: "45a348d0-e1f2-4966-a14f-5034036bc7c2"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:11:36.301433 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.301401    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45a348d0-e1f2-4966-a14f-5034036bc7c2-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:11:36.301433 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.301429    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n55pk\" (UniqueName: \"kubernetes.io/projected/45a348d0-e1f2-4966-a14f-5034036bc7c2-kube-api-access-n55pk\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:11:36.301681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.301465    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:11:36.312422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.312399    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:11:36.353554 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.353515    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a348d0-e1f2-4966-a14f-5034036bc7c2-plugins" pod "45a348d0-e1f2-4966-a14f-5034036bc7c2" (UID: "45a348d0-e1f2-4966-a14f-5034036bc7c2"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:11:36.402443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.402406    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/45a348d0-e1f2-4966-a14f-5034036bc7c2-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:11:36.402443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.402437    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:11:36.917078 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.917043    5746 generic.go:356] "Generic (PLEG): container finished" podID="45a348d0-e1f2-4966-a14f-5034036bc7c2" containerID="b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8" exitCode=0
Oct 30 18:11:36.917521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.917108    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:11:36.917521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.917131    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"45a348d0-e1f2-4966-a14f-5034036bc7c2","Type":"ContainerDied","Data":"b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8"}
Oct 30 18:11:36.917521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.917176    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"45a348d0-e1f2-4966-a14f-5034036bc7c2","Type":"ContainerDied","Data":"748520b5b5fd5a2b01c8d5072d30a846a7322abb40851367b67a57e734207a5e"}
Oct 30 18:11:36.917521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.917192    5746 scope.go:122] "RemoveContainer" containerID="b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8"
Oct 30 18:11:36.925892 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.925872    5746 scope.go:122] "RemoveContainer" containerID="b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8"
Oct 30 18:11:36.926181 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:11:36.926165    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8\": container with ID starting with b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8 not found: ID does not exist" containerID="b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8"
Oct 30 18:11:36.926227 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.926190    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8"} err="failed to get container status \"b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8\": rpc error: code = NotFound desc = could not find container \"b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8\": container with ID starting with b48f8c92a108482e129ea5e42c637b728037106da4e10c2d96452fd99c40eee8 not found: ID does not exist"
Oct 30 18:11:36.966243 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.966215    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:11:36.977898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:36.977869    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:11:38.007703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:38.007674    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45a348d0-e1f2-4966-a14f-5034036bc7c2" path="/var/lib/kubelet/pods/45a348d0-e1f2-4966-a14f-5034036bc7c2/volumes"
Oct 30 18:11:41.642729 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.642697    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"]
Oct 30 18:11:41.643241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.642996    5746 cpu_manager.go:414] "containerMap: removing container" podUID="45a348d0-e1f2-4966-a14f-5034036bc7c2" containerName="perses"
Oct 30 18:11:41.643241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.643007    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45a348d0-e1f2-4966-a14f-5034036bc7c2" containerName="perses"
Oct 30 18:11:41.643241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.643061    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="45a348d0-e1f2-4966-a14f-5034036bc7c2" containerName="perses"
Oct 30 18:11:41.664562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.664535    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:41.666680 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.666656    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"]
Oct 30 18:11:41.678885 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.678863    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:11:41.679061 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.679047    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-nk4cv\""
Oct 30 18:11:41.690661 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.690643    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:11:41.690661 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.690655    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:11:41.690804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.690649    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:11:41.745492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.745458    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-ta-internal\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:41.745651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.745517    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:41.745651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.745625    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9pvf\" (UniqueName: \"kubernetes.io/projected/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-kube-api-access-x9pvf\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:41.846523 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.846484    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:41.846718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.846538    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9pvf\" (UniqueName: \"kubernetes.io/projected/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-kube-api-access-x9pvf\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:41.846718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.846587    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-ta-internal\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:41.846718 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:11:41.846636    5746 secret.go:189] Couldn't get secret redhat-ods-monitoring/data-science-collector-ta-server-cert: secret "data-science-collector-ta-server-cert" not found
Oct 30 18:11:41.846718 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:11:41.846713    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert podName:6b6c15ca-3c27-4ae3-8d76-05fcda9158f2 nodeName:}" failed. No retries permitted until 2026-10-30 18:11:42.34669648 +0000 UTC m=+872.513514585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "data-science-collector-ta-server-cert" (UniqueName: "kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert") pod "data-science-collector-targetallocator-5d888db666-d75vb" (UID: "6b6c15ca-3c27-4ae3-8d76-05fcda9158f2") : secret "data-science-collector-ta-server-cert" not found
Oct 30 18:11:41.847264 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.847240    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-ta-internal\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:41.875520 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:41.875489    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9pvf\" (UniqueName: \"kubernetes.io/projected/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-kube-api-access-x9pvf\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:42.350622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:42.350579    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:42.350774 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:11:42.350709    5746 secret.go:189] Couldn't get secret redhat-ods-monitoring/data-science-collector-ta-server-cert: secret "data-science-collector-ta-server-cert" not found
Oct 30 18:11:42.350774 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:11:42.350767    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert podName:6b6c15ca-3c27-4ae3-8d76-05fcda9158f2 nodeName:}" failed. No retries permitted until 2026-10-30 18:11:43.350754196 +0000 UTC m=+873.517572313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "data-science-collector-ta-server-cert" (UniqueName: "kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert") pod "data-science-collector-targetallocator-5d888db666-d75vb" (UID: "6b6c15ca-3c27-4ae3-8d76-05fcda9158f2") : secret "data-science-collector-ta-server-cert" not found
Oct 30 18:11:42.789084 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:11:42.789004    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:11:43.359455 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:43.359406    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:43.361986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:43.361967    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5d888db666-d75vb\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:43.478085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:43.478050    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:43.775539 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:43.775453    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"]
Oct 30 18:11:43.779153 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:11:43.779128    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b6c15ca_3c27_4ae3_8d76_05fcda9158f2.slice/crio-a149771042fd41b2028c936e739b9fc7634fbf0526ba1eab8959aeaaad7114cf WatchSource:0}: Error finding container a149771042fd41b2028c936e739b9fc7634fbf0526ba1eab8959aeaaad7114cf: Status 404 returned error can't find the container with id a149771042fd41b2028c936e739b9fc7634fbf0526ba1eab8959aeaaad7114cf
Oct 30 18:11:43.946236 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:43.946200    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb" event={"ID":"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2","Type":"ContainerStarted","Data":"a149771042fd41b2028c936e739b9fc7634fbf0526ba1eab8959aeaaad7114cf"}
Oct 30 18:11:46.549198 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:46.549154    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:11:46.549579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:46.549232    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:11:46.959966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:46.959930    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb" event={"ID":"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2","Type":"ContainerStarted","Data":"01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73"}
Oct 30 18:11:46.960160 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:46.960100    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:46.961682 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:46.961661    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:47.000830 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:47.000666    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb" podStartSLOduration=3.233297673 podStartE2EDuration="6.000649897s" totalImagesPullingTime="2.767352224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:11:47.000002064 +0000 UTC m=+877.166820211" watchObservedRunningTime="2026-10-30 18:11:47.000649897 +0000 UTC m=+877.167468027"
Oct 30 18:11:56.955025 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:56.954994    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"]
Oct 30 18:11:56.955461 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:56.955188    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb" podUID="6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" containerName="ta-container" containerID="cri-o://01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73" gracePeriod=30
Oct 30 18:11:56.961157 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:56.961123    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb" podUID="6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.19:8080/readyz\": dial tcp 10.133.0.19:8080: connect: connection refused"
Oct 30 18:11:57.195688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.195662    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:57.373352 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.373322    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-kube-api-access-x9pvf\" (UniqueName: \"kubernetes.io/projected/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-kube-api-access-x9pvf\") pod \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") "
Oct 30 18:11:57.373524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.373364    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-ta-internal\" (UniqueName: \"kubernetes.io/configmap/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-ta-internal\") pod \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") "
Oct 30 18:11:57.373524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.373442    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert\") pod \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\" (UID: \"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2\") "
Oct 30 18:11:57.373802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.373767    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-ta-internal" pod "6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" (UID: "6b6c15ca-3c27-4ae3-8d76-05fcda9158f2"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:11:57.376053 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.376022    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert" pod "6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" (UID: "6b6c15ca-3c27-4ae3-8d76-05fcda9158f2"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:11:57.376053 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.376037    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-kube-api-access-x9pvf" pod "6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" (UID: "6b6c15ca-3c27-4ae3-8d76-05fcda9158f2"). InnerVolumeSpecName "kube-api-access-x9pvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:11:57.474531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.474487    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:11:57.474531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.474523    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9pvf\" (UniqueName: \"kubernetes.io/projected/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-kube-api-access-x9pvf\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:11:57.474531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:57.474538    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:11:58.006873 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.006842    5746 generic.go:356] "Generic (PLEG): container finished" podID="6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" containerID="01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73" exitCode=0
Oct 30 18:11:58.007349 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.006929    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"
Oct 30 18:11:58.007686 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.007664    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb" event={"ID":"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2","Type":"ContainerDied","Data":"01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73"}
Oct 30 18:11:58.007734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.007698    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb" event={"ID":"6b6c15ca-3c27-4ae3-8d76-05fcda9158f2","Type":"ContainerDied","Data":"a149771042fd41b2028c936e739b9fc7634fbf0526ba1eab8959aeaaad7114cf"}
Oct 30 18:11:58.007734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.007715    5746 scope.go:122] "RemoveContainer" containerID="01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73"
Oct 30 18:11:58.015755 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.015736    5746 scope.go:122] "RemoveContainer" containerID="01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73"
Oct 30 18:11:58.016063 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:11:58.016040    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73\": container with ID starting with 01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73 not found: ID does not exist" containerID="01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73"
Oct 30 18:11:58.016159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.016075    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73"} err="failed to get container status \"01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73\": rpc error: code = NotFound desc = could not find container \"01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73\": container with ID starting with 01c727ec14b4b44f2695b6e58088e1746c195a15fd337cec14fb2f5779c9bf73 not found: ID does not exist"
Oct 30 18:11:58.059093 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.059068    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"]
Oct 30 18:11:58.070428 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:11:58.070401    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d888db666-d75vb"]
Oct 30 18:12:00.013544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:00.013509    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" path="/var/lib/kubelet/pods/6b6c15ca-3c27-4ae3-8d76-05fcda9158f2/volumes"
Oct 30 18:12:12.790776 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:12:12.790732    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:12:21.646259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.646166    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"]
Oct 30 18:12:21.646675 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.646532    5746 cpu_manager.go:414] "containerMap: removing container" podUID="6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" containerName="ta-container"
Oct 30 18:12:21.646675 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.646549    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" containerName="ta-container"
Oct 30 18:12:21.646756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.646693    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b6c15ca-3c27-4ae3-8d76-05fcda9158f2" containerName="ta-container"
Oct 30 18:12:21.651055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.651036    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.677133 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.677109    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:12:21.677133 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.677130    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-g4zjw\""
Oct 30 18:12:21.678220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.678197    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"]
Oct 30 18:12:21.749869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.749835    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/248685d7-5a70-4955-941e-5a1ef05708fd-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.750068 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.749884    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/248685d7-5a70-4955-941e-5a1ef05708fd-ta-internal\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.750068 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.749995    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86jg9\" (UniqueName: \"kubernetes.io/projected/248685d7-5a70-4955-941e-5a1ef05708fd-kube-api-access-86jg9\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.756626 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.756610    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:12:21.756796 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.756770    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:12:21.756860 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.756826    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:12:21.850873 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.850837    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86jg9\" (UniqueName: \"kubernetes.io/projected/248685d7-5a70-4955-941e-5a1ef05708fd-kube-api-access-86jg9\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.851063 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.850886    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/248685d7-5a70-4955-941e-5a1ef05708fd-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.851063 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.850951    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/248685d7-5a70-4955-941e-5a1ef05708fd-ta-internal\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.851552 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.851532    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/248685d7-5a70-4955-941e-5a1ef05708fd-ta-internal\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.853489 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.853462    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/248685d7-5a70-4955-941e-5a1ef05708fd-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.873877 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.873854    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86jg9\" (UniqueName: \"kubernetes.io/projected/248685d7-5a70-4955-941e-5a1ef05708fd-kube-api-access-86jg9\") pod \"data-science-collector-targetallocator-69d44dccf8-ckx6z\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:21.963026 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:21.962954    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:22.284174 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:22.284095    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"]
Oct 30 18:12:22.287749 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:12:22.287713    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod248685d7_5a70_4955_941e_5a1ef05708fd.slice/crio-81e6a2dd7e9c66cda99350883069e83dd7ef235b7045ac74010088386f7aefde WatchSource:0}: Error finding container 81e6a2dd7e9c66cda99350883069e83dd7ef235b7045ac74010088386f7aefde: Status 404 returned error can't find the container with id 81e6a2dd7e9c66cda99350883069e83dd7ef235b7045ac74010088386f7aefde
Oct 30 18:12:22.289964 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:22.289932    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:12:22.290045 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:22.289992    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:12:23.098694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:23.098658    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z" event={"ID":"248685d7-5a70-4955-941e-5a1ef05708fd","Type":"ContainerStarted","Data":"0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e"}
Oct 30 18:12:23.098694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:23.098701    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z" event={"ID":"248685d7-5a70-4955-941e-5a1ef05708fd","Type":"ContainerStarted","Data":"81e6a2dd7e9c66cda99350883069e83dd7ef235b7045ac74010088386f7aefde"}
Oct 30 18:12:23.099228 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:23.098927    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:23.100510 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:23.100489    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:23.138627 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:23.138575    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z" podStartSLOduration=2.138562833 podStartE2EDuration="2.138562833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:12:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:12:23.138257411 +0000 UTC m=+913.305075535" watchObservedRunningTime="2026-10-30 18:12:23.138562833 +0000 UTC m=+913.305380954"
Oct 30 18:12:37.191210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.191173    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"]
Oct 30 18:12:37.191754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.191398    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z" podUID="248685d7-5a70-4955-941e-5a1ef05708fd" containerName="ta-container" containerID="cri-o://0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e" gracePeriod=30
Oct 30 18:12:37.433389 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.433362    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:37.468017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.467948    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/248685d7-5a70-4955-941e-5a1ef05708fd-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/248685d7-5a70-4955-941e-5a1ef05708fd-data-science-collector-ta-server-cert\") pod \"248685d7-5a70-4955-941e-5a1ef05708fd\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") "
Oct 30 18:12:37.468148 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.468032    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/248685d7-5a70-4955-941e-5a1ef05708fd-kube-api-access-86jg9\" (UniqueName: \"kubernetes.io/projected/248685d7-5a70-4955-941e-5a1ef05708fd-kube-api-access-86jg9\") pod \"248685d7-5a70-4955-941e-5a1ef05708fd\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") "
Oct 30 18:12:37.468148 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.468116    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/248685d7-5a70-4955-941e-5a1ef05708fd-ta-internal\" (UniqueName: \"kubernetes.io/configmap/248685d7-5a70-4955-941e-5a1ef05708fd-ta-internal\") pod \"248685d7-5a70-4955-941e-5a1ef05708fd\" (UID: \"248685d7-5a70-4955-941e-5a1ef05708fd\") "
Oct 30 18:12:37.468462 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.468437    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/248685d7-5a70-4955-941e-5a1ef05708fd-ta-internal" pod "248685d7-5a70-4955-941e-5a1ef05708fd" (UID: "248685d7-5a70-4955-941e-5a1ef05708fd"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:12:37.470249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.470226    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/248685d7-5a70-4955-941e-5a1ef05708fd-data-science-collector-ta-server-cert" pod "248685d7-5a70-4955-941e-5a1ef05708fd" (UID: "248685d7-5a70-4955-941e-5a1ef05708fd"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:12:37.470494 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.470468    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/248685d7-5a70-4955-941e-5a1ef05708fd-kube-api-access-86jg9" pod "248685d7-5a70-4955-941e-5a1ef05708fd" (UID: "248685d7-5a70-4955-941e-5a1ef05708fd"). InnerVolumeSpecName "kube-api-access-86jg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:12:37.569166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.569135    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86jg9\" (UniqueName: \"kubernetes.io/projected/248685d7-5a70-4955-941e-5a1ef05708fd-kube-api-access-86jg9\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:12:37.569166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.569161    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/248685d7-5a70-4955-941e-5a1ef05708fd-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:12:37.569166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:37.569171    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/248685d7-5a70-4955-941e-5a1ef05708fd-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:12:38.157037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.157002    5746 generic.go:356] "Generic (PLEG): container finished" podID="248685d7-5a70-4955-941e-5a1ef05708fd" containerID="0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e" exitCode=0
Oct 30 18:12:38.157210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.157043    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"
Oct 30 18:12:38.157210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.157048    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z" event={"ID":"248685d7-5a70-4955-941e-5a1ef05708fd","Type":"ContainerDied","Data":"0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e"}
Oct 30 18:12:38.157210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.157147    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z" event={"ID":"248685d7-5a70-4955-941e-5a1ef05708fd","Type":"ContainerDied","Data":"81e6a2dd7e9c66cda99350883069e83dd7ef235b7045ac74010088386f7aefde"}
Oct 30 18:12:38.157210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.157165    5746 scope.go:122] "RemoveContainer" containerID="0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e"
Oct 30 18:12:38.165158 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.165133    5746 scope.go:122] "RemoveContainer" containerID="0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e"
Oct 30 18:12:38.165448 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:12:38.165428    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e\": container with ID starting with 0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e not found: ID does not exist" containerID="0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e"
Oct 30 18:12:38.165535 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.165453    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e"} err="failed to get container status \"0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e\": rpc error: code = NotFound desc = could not find container \"0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e\": container with ID starting with 0307b01773002f9c2fe3ac9d379a6dd3c1b485ed561a1c0752e51ea66b480a1e not found: ID does not exist"
Oct 30 18:12:38.207124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.207091    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"]
Oct 30 18:12:38.218681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:38.218652    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-69d44dccf8-ckx6z"]
Oct 30 18:12:40.009899 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:40.009862    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="248685d7-5a70-4955-941e-5a1ef05708fd" path="/var/lib/kubelet/pods/248685d7-5a70-4955-941e-5a1ef05708fd/volumes"
Oct 30 18:12:42.789805 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:12:42.789774    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:12:46.954699 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.954661    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"]
Oct 30 18:12:46.955122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.955011    5746 cpu_manager.go:414] "containerMap: removing container" podUID="248685d7-5a70-4955-941e-5a1ef05708fd" containerName="ta-container"
Oct 30 18:12:46.955122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.955024    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="248685d7-5a70-4955-941e-5a1ef05708fd" containerName="ta-container"
Oct 30 18:12:46.955122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.955103    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="248685d7-5a70-4955-941e-5a1ef05708fd" containerName="ta-container"
Oct 30 18:12:46.959273 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.959257    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"
Oct 30 18:12:46.977050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.976739    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:12:46.977444 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.977420    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"]
Oct 30 18:12:46.988434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.988406    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:12:46.988536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:46.988500    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:12:47.038057 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:47.038022    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4jb6\" (UniqueName: \"kubernetes.io/projected/fba67e1b-d105-421c-8311-97de0de3959b-kube-api-access-c4jb6\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6\" (UID: \"fba67e1b-d105-421c-8311-97de0de3959b\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"
Oct 30 18:12:47.139019 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:47.138973    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4jb6\" (UniqueName: \"kubernetes.io/projected/fba67e1b-d105-421c-8311-97de0de3959b-kube-api-access-c4jb6\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6\" (UID: \"fba67e1b-d105-421c-8311-97de0de3959b\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"
Oct 30 18:12:47.163247 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:47.163214    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4jb6\" (UniqueName: \"kubernetes.io/projected/fba67e1b-d105-421c-8311-97de0de3959b-kube-api-access-c4jb6\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6\" (UID: \"fba67e1b-d105-421c-8311-97de0de3959b\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"
Oct 30 18:12:47.270738 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:47.270656    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"
Oct 30 18:12:47.566228 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:47.566144    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"]
Oct 30 18:12:47.569868 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:12:47.569822    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfba67e1b_d105_421c_8311_97de0de3959b.slice/crio-7d708a081bf3e7f8a05176f76d4e44392fe2909c693800b4f08be601c04a2589 WatchSource:0}: Error finding container 7d708a081bf3e7f8a05176f76d4e44392fe2909c693800b4f08be601c04a2589: Status 404 returned error can't find the container with id 7d708a081bf3e7f8a05176f76d4e44392fe2909c693800b4f08be601c04a2589
Oct 30 18:12:47.571856 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:47.571835    5746 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:12:47.989841 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:47.989807    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:12:47.994269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:47.994248    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.014333 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.014310    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:12:48.046587 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.046559    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/78b5d2e0-3100-4c78-8e62-9581889c49cc-plugins\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.046734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.046592    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l94f\" (UniqueName: \"kubernetes.io/projected/78b5d2e0-3100-4c78-8e62-9581889c49cc-kube-api-access-7l94f\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.046778 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.046756    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78b5d2e0-3100-4c78-8e62-9581889c49cc-config\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.060433 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.060410    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:12:48.148085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.148045    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/78b5d2e0-3100-4c78-8e62-9581889c49cc-plugins\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.148085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.148082    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l94f\" (UniqueName: \"kubernetes.io/projected/78b5d2e0-3100-4c78-8e62-9581889c49cc-kube-api-access-7l94f\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.148353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.148111    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78b5d2e0-3100-4c78-8e62-9581889c49cc-config\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.148521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.148499    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/78b5d2e0-3100-4c78-8e62-9581889c49cc-plugins\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.148759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.148743    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78b5d2e0-3100-4c78-8e62-9581889c49cc-config\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.171585 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.171557    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l94f\" (UniqueName: \"kubernetes.io/projected/78b5d2e0-3100-4c78-8e62-9581889c49cc-kube-api-access-7l94f\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:48.204601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.204563    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6" event={"ID":"fba67e1b-d105-421c-8311-97de0de3959b","Type":"ContainerStarted","Data":"7d708a081bf3e7f8a05176f76d4e44392fe2909c693800b4f08be601c04a2589"}
Oct 30 18:12:48.233164 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.233134    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"]
Oct 30 18:12:48.236742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.236727    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.248587 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.248568    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrfh4\" (UniqueName: \"kubernetes.io/projected/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-kube-api-access-rrfh4\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.248709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.248594    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-ta-internal\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.248709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.248628    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.251018 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.250998    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:12:48.251130 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.251078    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-mm5md\""
Oct 30 18:12:48.251130 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.251094    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:12:48.259411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.259383    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"]
Oct 30 18:12:48.349416 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.349383    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrfh4\" (UniqueName: \"kubernetes.io/projected/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-kube-api-access-rrfh4\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.349416 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.349413    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-ta-internal\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.349621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.349449    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.350179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.350142    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-ta-internal\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.352080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.352059    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.372258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.372238    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrfh4\" (UniqueName: \"kubernetes.io/projected/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-kube-api-access-rrfh4\") pod \"data-science-collector-targetallocator-5ff5d88b75-cjpqc\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.548273 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.548183    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:48.853330 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.852849    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"]
Oct 30 18:12:48.857842 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:12:48.857812    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a1a1d90_8fa9_4d2f_a677_6ff9037c110a.slice/crio-bd920e060fdd381fb02da9446ba36264a268981cc3d9e549e3b5e7e1fd20d053 WatchSource:0}: Error finding container bd920e060fdd381fb02da9446ba36264a268981cc3d9e549e3b5e7e1fd20d053: Status 404 returned error can't find the container with id bd920e060fdd381fb02da9446ba36264a268981cc3d9e549e3b5e7e1fd20d053
Oct 30 18:12:48.860287 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.860250    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:12:48.860385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:48.860320    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:12:49.211139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:49.211064    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc" event={"ID":"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a","Type":"ContainerStarted","Data":"4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e"}
Oct 30 18:12:49.211139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:49.211109    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc" event={"ID":"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a","Type":"ContainerStarted","Data":"bd920e060fdd381fb02da9446ba36264a268981cc3d9e549e3b5e7e1fd20d053"}
Oct 30 18:12:49.211563 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:49.211290    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:49.212516 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:49.212486    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc" podUID="7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.23:8080/readyz\": dial tcp 10.133.0.23:8080: connect: connection refused"
Oct 30 18:12:49.253829 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:49.253779    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc" podStartSLOduration=1.253762765 podStartE2EDuration="1.253762765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:12:49.252223354 +0000 UTC m=+939.419041506" watchObservedRunningTime="2026-10-30 18:12:49.253762765 +0000 UTC m=+939.420580896"
Oct 30 18:12:50.219153 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.219056    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6" event={"ID":"fba67e1b-d105-421c-8311-97de0de3959b","Type":"ContainerStarted","Data":"40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968"}
Oct 30 18:12:50.221143 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.221119    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:12:50.257764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.257720    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6" podStartSLOduration=1.8132822050000001 podStartE2EDuration="4.257706127s" totalImagesPullingTime="2.444423922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:12:50.256846677 +0000 UTC m=+940.423664805" watchObservedRunningTime="2026-10-30 18:12:50.257706127 +0000 UTC m=+940.424524232"
Oct 30 18:12:50.469013 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.468977    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:50.497509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.497443    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:50.570348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.570315    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:50.570512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.570417    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:50.584733 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.584712    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:50.687714 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.687686    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:50.711566 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:50.711540    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:51.005660 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:51.005617    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:51.299710 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:51.299620    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:12:51.303547 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:12:51.303523    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78b5d2e0_3100_4c78_8e62_9581889c49cc.slice/crio-41111893cd1851fb16954587872c51edaac57abd12eafb1f10cc651a01155dbd WatchSource:0}: Error finding container 41111893cd1851fb16954587872c51edaac57abd12eafb1f10cc651a01155dbd: Status 404 returned error can't find the container with id 41111893cd1851fb16954587872c51edaac57abd12eafb1f10cc651a01155dbd
Oct 30 18:12:52.229245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:52.229210    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"78b5d2e0-3100-4c78-8e62-9581889c49cc","Type":"ContainerStarted","Data":"8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3"}
Oct 30 18:12:52.229245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:52.229253    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"78b5d2e0-3100-4c78-8e62-9581889c49cc","Type":"ContainerStarted","Data":"41111893cd1851fb16954587872c51edaac57abd12eafb1f10cc651a01155dbd"}
Oct 30 18:12:52.229524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:52.229335    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:12:58.640398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:58.640333    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=11.640318824 podStartE2EDuration="11.640318824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:12:52.271297367 +0000 UTC m=+942.438115514" watchObservedRunningTime="2026-10-30 18:12:58.640318824 +0000 UTC m=+948.807136951"
Oct 30 18:12:58.640890 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:58.640833    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"]
Oct 30 18:12:58.641050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:58.641029    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6" podUID="fba67e1b-d105-421c-8311-97de0de3959b" containerName="thanos-querier" containerID="cri-o://40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968" gracePeriod=30
Oct 30 18:12:58.881651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:58.881628    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"
Oct 30 18:12:59.035289 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.035199    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fba67e1b-d105-421c-8311-97de0de3959b-kube-api-access-c4jb6\" (UniqueName: \"kubernetes.io/projected/fba67e1b-d105-421c-8311-97de0de3959b-kube-api-access-c4jb6\") pod \"fba67e1b-d105-421c-8311-97de0de3959b\" (UID: \"fba67e1b-d105-421c-8311-97de0de3959b\") "
Oct 30 18:12:59.037446 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.037414    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fba67e1b-d105-421c-8311-97de0de3959b-kube-api-access-c4jb6" pod "fba67e1b-d105-421c-8311-97de0de3959b" (UID: "fba67e1b-d105-421c-8311-97de0de3959b"). InnerVolumeSpecName "kube-api-access-c4jb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:12:59.136704 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.136661    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4jb6\" (UniqueName: \"kubernetes.io/projected/fba67e1b-d105-421c-8311-97de0de3959b-kube-api-access-c4jb6\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:12:59.258614 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.258578    5746 generic.go:356] "Generic (PLEG): container finished" podID="fba67e1b-d105-421c-8311-97de0de3959b" containerID="40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968" exitCode=0
Oct 30 18:12:59.258614 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.258616    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"
Oct 30 18:12:59.258812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.258639    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6" event={"ID":"fba67e1b-d105-421c-8311-97de0de3959b","Type":"ContainerDied","Data":"40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968"}
Oct 30 18:12:59.258812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.258674    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6" event={"ID":"fba67e1b-d105-421c-8311-97de0de3959b","Type":"ContainerDied","Data":"7d708a081bf3e7f8a05176f76d4e44392fe2909c693800b4f08be601c04a2589"}
Oct 30 18:12:59.258812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.258691    5746 scope.go:122] "RemoveContainer" containerID="40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968"
Oct 30 18:12:59.266589 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.266572    5746 scope.go:122] "RemoveContainer" containerID="40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968"
Oct 30 18:12:59.266855 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:12:59.266838    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968\": container with ID starting with 40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968 not found: ID does not exist" containerID="40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968"
Oct 30 18:12:59.266897 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.266864    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968"} err="failed to get container status \"40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968\": rpc error: code = NotFound desc = could not find container \"40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968\": container with ID starting with 40d1714ec871baf0c77e827e274c6906c35dfb770d2b3ecd563d806f31a5d968 not found: ID does not exist"
Oct 30 18:12:59.306231 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.306140    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"]
Oct 30 18:12:59.312289 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:12:59.312267    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mm5l6"]
Oct 30 18:13:00.008155 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:00.008109    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fba67e1b-d105-421c-8311-97de0de3959b" path="/var/lib/kubelet/pods/fba67e1b-d105-421c-8311-97de0de3959b/volumes"
Oct 30 18:13:03.236505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:03.236474    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:12.789605 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:13:12.789567    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:13:27.950362 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:27.949952    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"]
Oct 30 18:13:27.950362 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:27.950184    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc" podUID="7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" containerName="ta-container" containerID="cri-o://4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e" gracePeriod=30
Oct 30 18:13:28.057040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.057006    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:13:28.057312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.057261    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="78b5d2e0-3100-4c78-8e62-9581889c49cc" containerName="perses" containerID="cri-o://8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3" gracePeriod=30
Oct 30 18:13:28.204116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.204044    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:13:28.276308 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.276278    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-kube-api-access-rrfh4\" (UniqueName: \"kubernetes.io/projected/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-kube-api-access-rrfh4\") pod \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") "
Oct 30 18:13:28.276476 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.276324    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-data-science-collector-ta-server-cert\") pod \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") "
Oct 30 18:13:28.276476 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.276392    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-ta-internal\" (UniqueName: \"kubernetes.io/configmap/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-ta-internal\") pod \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\" (UID: \"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a\") "
Oct 30 18:13:28.276779 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.276755    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-ta-internal" pod "7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" (UID: "7a1a1d90-8fa9-4d2f-a677-6ff9037c110a"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:13:28.278591 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.278556    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-kube-api-access-rrfh4" pod "7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" (UID: "7a1a1d90-8fa9-4d2f-a677-6ff9037c110a"). InnerVolumeSpecName "kube-api-access-rrfh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:13:28.279087 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.279061    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-data-science-collector-ta-server-cert" pod "7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" (UID: "7a1a1d90-8fa9-4d2f-a677-6ff9037c110a"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:13:28.309597 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.309571    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:28.368557 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.368524    5746 generic.go:356] "Generic (PLEG): container finished" podID="78b5d2e0-3100-4c78-8e62-9581889c49cc" containerID="8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3" exitCode=0
Oct 30 18:13:28.368745 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.368575    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:28.368745 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.368608    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"78b5d2e0-3100-4c78-8e62-9581889c49cc","Type":"ContainerDied","Data":"8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3"}
Oct 30 18:13:28.368745 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.368648    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"78b5d2e0-3100-4c78-8e62-9581889c49cc","Type":"ContainerDied","Data":"41111893cd1851fb16954587872c51edaac57abd12eafb1f10cc651a01155dbd"}
Oct 30 18:13:28.368745 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.368669    5746 scope.go:122] "RemoveContainer" containerID="8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3"
Oct 30 18:13:28.370384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.370278    5746 generic.go:356] "Generic (PLEG): container finished" podID="7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" containerID="4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e" exitCode=0
Oct 30 18:13:28.370384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.370339    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"
Oct 30 18:13:28.370721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.370336    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc" event={"ID":"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a","Type":"ContainerDied","Data":"4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e"}
Oct 30 18:13:28.370944 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.370891    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc" event={"ID":"7a1a1d90-8fa9-4d2f-a677-6ff9037c110a","Type":"ContainerDied","Data":"bd920e060fdd381fb02da9446ba36264a268981cc3d9e549e3b5e7e1fd20d053"}
Oct 30 18:13:28.377217 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.377193    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78b5d2e0-3100-4c78-8e62-9581889c49cc-kube-api-access-7l94f\" (UniqueName: \"kubernetes.io/projected/78b5d2e0-3100-4c78-8e62-9581889c49cc-kube-api-access-7l94f\") pod \"78b5d2e0-3100-4c78-8e62-9581889c49cc\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") "
Oct 30 18:13:28.377334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.377241    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78b5d2e0-3100-4c78-8e62-9581889c49cc-config\" (UniqueName: \"kubernetes.io/configmap/78b5d2e0-3100-4c78-8e62-9581889c49cc-config\") pod \"78b5d2e0-3100-4c78-8e62-9581889c49cc\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") "
Oct 30 18:13:28.377334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.377330    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"78b5d2e0-3100-4c78-8e62-9581889c49cc\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") "
Oct 30 18:13:28.377450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.377374    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78b5d2e0-3100-4c78-8e62-9581889c49cc-plugins\" (UniqueName: \"kubernetes.io/empty-dir/78b5d2e0-3100-4c78-8e62-9581889c49cc-plugins\") pod \"78b5d2e0-3100-4c78-8e62-9581889c49cc\" (UID: \"78b5d2e0-3100-4c78-8e62-9581889c49cc\") "
Oct 30 18:13:28.377552 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.377535    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:13:28.377610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.377569    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrfh4\" (UniqueName: \"kubernetes.io/projected/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-kube-api-access-rrfh4\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:13:28.377610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.377585    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:13:28.377961 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.377711    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78b5d2e0-3100-4c78-8e62-9581889c49cc-config" pod "78b5d2e0-3100-4c78-8e62-9581889c49cc" (UID: "78b5d2e0-3100-4c78-8e62-9581889c49cc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:13:28.378266 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.378250    5746 scope.go:122] "RemoveContainer" containerID="8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3"
Oct 30 18:13:28.378604 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:13:28.378581    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3\": container with ID starting with 8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3 not found: ID does not exist" containerID="8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3"
Oct 30 18:13:28.378690 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.378611    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3"} err="failed to get container status \"8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3\": rpc error: code = NotFound desc = could not find container \"8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3\": container with ID starting with 8205e31b14961ab8c77061639d62259be422c28f4ae0eb5cdd9258fd7d1bb3a3 not found: ID does not exist"
Oct 30 18:13:28.378690 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.378627    5746 scope.go:122] "RemoveContainer" containerID="4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e"
Oct 30 18:13:28.380230 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.380131    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78b5d2e0-3100-4c78-8e62-9581889c49cc-kube-api-access-7l94f" pod "78b5d2e0-3100-4c78-8e62-9581889c49cc" (UID: "78b5d2e0-3100-4c78-8e62-9581889c49cc"). InnerVolumeSpecName "kube-api-access-7l94f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:13:28.380441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.380422    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "78b5d2e0-3100-4c78-8e62-9581889c49cc" (UID: "78b5d2e0-3100-4c78-8e62-9581889c49cc"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:13:28.386869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.386846    5746 scope.go:122] "RemoveContainer" containerID="4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e"
Oct 30 18:13:28.387187 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:13:28.387167    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e\": container with ID starting with 4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e not found: ID does not exist" containerID="4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e"
Oct 30 18:13:28.387244 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.387193    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e"} err="failed to get container status \"4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e\": rpc error: code = NotFound desc = could not find container \"4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e\": container with ID starting with 4065d5c1e4267cf1ad80555836d2253c9155ca0e4c0fb3b210076cf6d8700e0e not found: ID does not exist"
Oct 30 18:13:28.417605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.417574    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"]
Oct 30 18:13:28.421704 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.421672    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5ff5d88b75-cjpqc"]
Oct 30 18:13:28.478410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.478341    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l94f\" (UniqueName: \"kubernetes.io/projected/78b5d2e0-3100-4c78-8e62-9581889c49cc-kube-api-access-7l94f\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:13:28.478410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.478369    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78b5d2e0-3100-4c78-8e62-9581889c49cc-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:13:28.478410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.478400    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:13:28.491663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.491644    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:13:28.522528 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.522497    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78b5d2e0-3100-4c78-8e62-9581889c49cc-plugins" pod "78b5d2e0-3100-4c78-8e62-9581889c49cc" (UID: "78b5d2e0-3100-4c78-8e62-9581889c49cc"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:13:28.578773 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.578744    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:13:28.578773 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.578770    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/78b5d2e0-3100-4c78-8e62-9581889c49cc-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:13:28.711742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.711710    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:13:28.715723 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:28.715698    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:13:30.007880 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:30.007843    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78b5d2e0-3100-4c78-8e62-9581889c49cc" path="/var/lib/kubelet/pods/78b5d2e0-3100-4c78-8e62-9581889c49cc/volumes"
Oct 30 18:13:30.008320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:30.008227    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" path="/var/lib/kubelet/pods/7a1a1d90-8fa9-4d2f-a677-6ff9037c110a/volumes"
Oct 30 18:13:32.631427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631394    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631679    5746 cpu_manager.go:414] "containerMap: removing container" podUID="78b5d2e0-3100-4c78-8e62-9581889c49cc" containerName="perses"
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631690    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78b5d2e0-3100-4c78-8e62-9581889c49cc" containerName="perses"
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631701    5746 cpu_manager.go:414] "containerMap: removing container" podUID="fba67e1b-d105-421c-8311-97de0de3959b" containerName="thanos-querier"
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631706    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fba67e1b-d105-421c-8311-97de0de3959b" containerName="thanos-querier"
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631720    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" containerName="ta-container"
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631726    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" containerName="ta-container"
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631773    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a1a1d90-8fa9-4d2f-a677-6ff9037c110a" containerName="ta-container"
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631783    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="78b5d2e0-3100-4c78-8e62-9581889c49cc" containerName="perses"
Oct 30 18:13:32.631803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.631790    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="fba67e1b-d105-421c-8311-97de0de3959b" containerName="thanos-querier"
Oct 30 18:13:32.653122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.653094    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.655114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.655092    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:13:32.754984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.754938    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:13:32.757322 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.757303    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:13:32.757419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.757324    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:13:32.757419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.757324    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:13:32.810641 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.810610    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/943e9e5a-59c5-4224-8de6-d79ecdad1f59-config\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.810754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.810655    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztjps\" (UniqueName: \"kubernetes.io/projected/943e9e5a-59c5-4224-8de6-d79ecdad1f59-kube-api-access-ztjps\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.810801 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.810785    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/943e9e5a-59c5-4224-8de6-d79ecdad1f59-plugins\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.911931 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.911830    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/943e9e5a-59c5-4224-8de6-d79ecdad1f59-plugins\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.911931 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.911874    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/943e9e5a-59c5-4224-8de6-d79ecdad1f59-config\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.911931 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.911899    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztjps\" (UniqueName: \"kubernetes.io/projected/943e9e5a-59c5-4224-8de6-d79ecdad1f59-kube-api-access-ztjps\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.912304 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.912281    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/943e9e5a-59c5-4224-8de6-d79ecdad1f59-plugins\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.912533 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.912514    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/943e9e5a-59c5-4224-8de6-d79ecdad1f59-config\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:32.935501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:32.935473    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztjps\" (UniqueName: \"kubernetes.io/projected/943e9e5a-59c5-4224-8de6-d79ecdad1f59-kube-api-access-ztjps\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:39.766407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:39.766370    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:39.795182 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:39.795154    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:39.867470 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:39.867435    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:39.867631 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:39.867606    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:39.881755 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:39.881730    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:39.985690 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:39.985653    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:40.013341 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:40.013317    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:40.167365 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:40.167337    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:40.462602 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:40.462533    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:13:40.465701 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:13:40.465672    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod943e9e5a_59c5_4224_8de6_d79ecdad1f59.slice/crio-51b41d9411138d2222cea6d0ccb2d79c4ff35eb2392b180ce4fecbca4d162774 WatchSource:0}: Error finding container 51b41d9411138d2222cea6d0ccb2d79c4ff35eb2392b180ce4fecbca4d162774: Status 404 returned error can't find the container with id 51b41d9411138d2222cea6d0ccb2d79c4ff35eb2392b180ce4fecbca4d162774
Oct 30 18:13:41.425332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:41.425291    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"943e9e5a-59c5-4224-8de6-d79ecdad1f59","Type":"ContainerStarted","Data":"3231c02bcc8a85f2d5049a7d544cb6d7c2a769e9016692f2a56c7be051aa0fcb"}
Oct 30 18:13:41.425332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:41.425335    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"943e9e5a-59c5-4224-8de6-d79ecdad1f59","Type":"ContainerStarted","Data":"51b41d9411138d2222cea6d0ccb2d79c4ff35eb2392b180ce4fecbca4d162774"}
Oct 30 18:13:41.425852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:41.425422    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:13:41.466071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:41.466022    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=9.466006738 podStartE2EDuration="9.466006738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:13:41.465557777 +0000 UTC m=+991.632375904" watchObservedRunningTime="2026-10-30 18:13:41.466006738 +0000 UTC m=+991.632824869"
Oct 30 18:13:42.788729 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:13:42.788689    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:13:52.432206 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:13:52.432176    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:12.789847 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:14:12.789806    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:14:22.374453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.374418    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:14:22.375243 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.375199    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="943e9e5a-59c5-4224-8de6-d79ecdad1f59" containerName="perses" containerID="cri-o://3231c02bcc8a85f2d5049a7d544cb6d7c2a769e9016692f2a56c7be051aa0fcb" gracePeriod=30
Oct 30 18:14:22.430965 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.430932    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-perses-0" podUID="943e9e5a-59c5-4224-8de6-d79ecdad1f59" containerName="perses" probeResult="failure" output="Get \"http://10.133.0.24:8080/metrics\": dial tcp 10.133.0.24:8080: connect: connection refused"
Oct 30 18:14:22.588406 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.588375    5746 generic.go:356] "Generic (PLEG): container finished" podID="943e9e5a-59c5-4224-8de6-d79ecdad1f59" containerID="3231c02bcc8a85f2d5049a7d544cb6d7c2a769e9016692f2a56c7be051aa0fcb" exitCode=0
Oct 30 18:14:22.588577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.588411    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"943e9e5a-59c5-4224-8de6-d79ecdad1f59","Type":"ContainerDied","Data":"3231c02bcc8a85f2d5049a7d544cb6d7c2a769e9016692f2a56c7be051aa0fcb"}
Oct 30 18:14:22.624814 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.624747    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:22.704444 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.704417    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/943e9e5a-59c5-4224-8de6-d79ecdad1f59-plugins\" (UniqueName: \"kubernetes.io/empty-dir/943e9e5a-59c5-4224-8de6-d79ecdad1f59-plugins\") pod \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") "
Oct 30 18:14:22.704613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.704479    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/943e9e5a-59c5-4224-8de6-d79ecdad1f59-config\" (UniqueName: \"kubernetes.io/configmap/943e9e5a-59c5-4224-8de6-d79ecdad1f59-config\") pod \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") "
Oct 30 18:14:22.704613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.704589    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") "
Oct 30 18:14:22.704707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.704651    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/943e9e5a-59c5-4224-8de6-d79ecdad1f59-kube-api-access-ztjps\" (UniqueName: \"kubernetes.io/projected/943e9e5a-59c5-4224-8de6-d79ecdad1f59-kube-api-access-ztjps\") pod \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\" (UID: \"943e9e5a-59c5-4224-8de6-d79ecdad1f59\") "
Oct 30 18:14:22.704882 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.704855    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/943e9e5a-59c5-4224-8de6-d79ecdad1f59-config" pod "943e9e5a-59c5-4224-8de6-d79ecdad1f59" (UID: "943e9e5a-59c5-4224-8de6-d79ecdad1f59"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:14:22.707676 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.707642    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/943e9e5a-59c5-4224-8de6-d79ecdad1f59-kube-api-access-ztjps" pod "943e9e5a-59c5-4224-8de6-d79ecdad1f59" (UID: "943e9e5a-59c5-4224-8de6-d79ecdad1f59"). InnerVolumeSpecName "kube-api-access-ztjps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:14:22.708084 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.708061    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "943e9e5a-59c5-4224-8de6-d79ecdad1f59" (UID: "943e9e5a-59c5-4224-8de6-d79ecdad1f59"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:14:22.805343 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.805316    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/943e9e5a-59c5-4224-8de6-d79ecdad1f59-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:14:22.805507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.805359    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:14:22.805507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.805371    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztjps\" (UniqueName: \"kubernetes.io/projected/943e9e5a-59c5-4224-8de6-d79ecdad1f59-kube-api-access-ztjps\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:14:22.818540 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.818518    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:14:22.851482 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.851453    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/943e9e5a-59c5-4224-8de6-d79ecdad1f59-plugins" pod "943e9e5a-59c5-4224-8de6-d79ecdad1f59" (UID: "943e9e5a-59c5-4224-8de6-d79ecdad1f59"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:14:22.905744 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.905684    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:14:22.905744 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:22.905707    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/943e9e5a-59c5-4224-8de6-d79ecdad1f59-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:14:23.594677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:23.594637    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"943e9e5a-59c5-4224-8de6-d79ecdad1f59","Type":"ContainerDied","Data":"51b41d9411138d2222cea6d0ccb2d79c4ff35eb2392b180ce4fecbca4d162774"}
Oct 30 18:14:23.594677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:23.594665    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:23.595227 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:23.594699    5746 scope.go:122] "RemoveContainer" containerID="3231c02bcc8a85f2d5049a7d544cb6d7c2a769e9016692f2a56c7be051aa0fcb"
Oct 30 18:14:23.643429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:23.643365    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:14:23.654981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:23.654959    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:14:24.009283 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:24.009206    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="943e9e5a-59c5-4224-8de6-d79ecdad1f59" path="/var/lib/kubelet/pods/943e9e5a-59c5-4224-8de6-d79ecdad1f59/volumes"
Oct 30 18:14:31.718805 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.718775    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"]
Oct 30 18:14:31.719278 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.719199    5746 cpu_manager.go:414] "containerMap: removing container" podUID="943e9e5a-59c5-4224-8de6-d79ecdad1f59" containerName="perses"
Oct 30 18:14:31.719278 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.719216    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="943e9e5a-59c5-4224-8de6-d79ecdad1f59" containerName="perses"
Oct 30 18:14:31.719355 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.719289    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="943e9e5a-59c5-4224-8de6-d79ecdad1f59" containerName="perses"
Oct 30 18:14:31.723984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.723816    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"
Oct 30 18:14:31.752363 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.752338    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:14:31.752363 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.752338    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:14:31.752529 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.752456    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:14:31.753049 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.753032    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"]
Oct 30 18:14:31.880491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.880455    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2trm\" (UniqueName: \"kubernetes.io/projected/05ae3ee5-62a1-4e54-9d18-62d1feabec32-kube-api-access-b2trm\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl\" (UID: \"05ae3ee5-62a1-4e54-9d18-62d1feabec32\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"
Oct 30 18:14:31.981799 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:31.981718    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2trm\" (UniqueName: \"kubernetes.io/projected/05ae3ee5-62a1-4e54-9d18-62d1feabec32-kube-api-access-b2trm\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl\" (UID: \"05ae3ee5-62a1-4e54-9d18-62d1feabec32\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"
Oct 30 18:14:32.004791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.004765    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2trm\" (UniqueName: \"kubernetes.io/projected/05ae3ee5-62a1-4e54-9d18-62d1feabec32-kube-api-access-b2trm\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl\" (UID: \"05ae3ee5-62a1-4e54-9d18-62d1feabec32\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"
Oct 30 18:14:32.035258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.035229    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"
Oct 30 18:14:32.327307 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.327222    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"]
Oct 30 18:14:32.330924 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:14:32.330883    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05ae3ee5_62a1_4e54_9d18_62d1feabec32.slice/crio-7f083d4799012e41b78b4c41cb83fc12b313207ed67dd0914c9c4c76707c8cf5 WatchSource:0}: Error finding container 7f083d4799012e41b78b4c41cb83fc12b313207ed67dd0914c9c4c76707c8cf5: Status 404 returned error can't find the container with id 7f083d4799012e41b78b4c41cb83fc12b313207ed67dd0914c9c4c76707c8cf5
Oct 30 18:14:32.631150 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.631112    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl" event={"ID":"05ae3ee5-62a1-4e54-9d18-62d1feabec32","Type":"ContainerStarted","Data":"fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242"}
Oct 30 18:14:32.631327 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.631162    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl" event={"ID":"05ae3ee5-62a1-4e54-9d18-62d1feabec32","Type":"ContainerStarted","Data":"7f083d4799012e41b78b4c41cb83fc12b313207ed67dd0914c9c4c76707c8cf5"}
Oct 30 18:14:32.670099 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.670050    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl" podStartSLOduration=1.670031803 podStartE2EDuration="1.670031803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:14:32.66897409 +0000 UTC m=+1042.835792218" watchObservedRunningTime="2026-10-30 18:14:32.670031803 +0000 UTC m=+1042.836849927"
Oct 30 18:14:32.853088 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.853060    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:14:32.856351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.856331    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:32.878015 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.877986    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:14:32.930862 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.930792    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"]
Oct 30 18:14:32.933846 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.933819    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:32.940012 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.939985    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:14:32.950800 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.950776    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-lpm62\""
Oct 30 18:14:32.950800 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.950790    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:14:32.954870 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.954834    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"]
Oct 30 18:14:32.990709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.990681    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmgng\" (UniqueName: \"kubernetes.io/projected/f7763c17-8cc7-47c6-bf3f-16a7fa146631-kube-api-access-lmgng\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:32.990842 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.990717    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7763c17-8cc7-47c6-bf3f-16a7fa146631-plugins\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:32.990842 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:32.990826    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7763c17-8cc7-47c6-bf3f-16a7fa146631-config\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:33.040164 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.040135    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:14:33.091427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.091393    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-ta-internal\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.091578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.091437    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7763c17-8cc7-47c6-bf3f-16a7fa146631-config\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:33.091578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.091461    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.091578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.091542    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slcwd\" (UniqueName: \"kubernetes.io/projected/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-kube-api-access-slcwd\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.091678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.091625    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmgng\" (UniqueName: \"kubernetes.io/projected/f7763c17-8cc7-47c6-bf3f-16a7fa146631-kube-api-access-lmgng\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:33.091678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.091653    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7763c17-8cc7-47c6-bf3f-16a7fa146631-plugins\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:33.091996 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.091981    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7763c17-8cc7-47c6-bf3f-16a7fa146631-plugins\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:33.092220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.092205    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7763c17-8cc7-47c6-bf3f-16a7fa146631-config\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:33.114368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.114336    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmgng\" (UniqueName: \"kubernetes.io/projected/f7763c17-8cc7-47c6-bf3f-16a7fa146631-kube-api-access-lmgng\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:33.192715 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.192623    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-ta-internal\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.192715 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.192671    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.192715 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.192691    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slcwd\" (UniqueName: \"kubernetes.io/projected/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-kube-api-access-slcwd\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.193404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.193315    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-ta-internal\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.195225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.195201    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.215867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.215843    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slcwd\" (UniqueName: \"kubernetes.io/projected/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-kube-api-access-slcwd\") pod \"data-science-collector-targetallocator-dd8cf967-ktkb6\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.245512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.245487    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.538582 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.538494    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"]
Oct 30 18:14:33.541924 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:14:33.541886    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0e7639e_c293_42a4_b8d8_887e1ef1f21d.slice/crio-646e10c5a098ec82a6bb6151706c72484ea4902a34728056fb4512e422a5d3f7 WatchSource:0}: Error finding container 646e10c5a098ec82a6bb6151706c72484ea4902a34728056fb4512e422a5d3f7: Status 404 returned error can't find the container with id 646e10c5a098ec82a6bb6151706c72484ea4902a34728056fb4512e422a5d3f7
Oct 30 18:14:33.544030 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.544001    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:14:33.544095 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.544057    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:14:33.636885 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.636851    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6" event={"ID":"c0e7639e-c293-42a4-b8d8-887e1ef1f21d","Type":"ContainerStarted","Data":"298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379"}
Oct 30 18:14:33.637089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.636898    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6" event={"ID":"c0e7639e-c293-42a4-b8d8-887e1ef1f21d","Type":"ContainerStarted","Data":"646e10c5a098ec82a6bb6151706c72484ea4902a34728056fb4512e422a5d3f7"}
Oct 30 18:14:33.637089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.637016    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:33.637941 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.637900    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6" podUID="c0e7639e-c293-42a4-b8d8-887e1ef1f21d" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.27:8080/readyz\": dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:14:33.677284 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:33.677183    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6" podStartSLOduration=1.6771571440000002 podStartE2EDuration="1.677157144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:14:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:14:33.676866981 +0000 UTC m=+1043.843685124" watchObservedRunningTime="2026-10-30 18:14:33.677157144 +0000 UTC m=+1043.843975270"
Oct 30 18:14:34.641196 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:34.641160    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:14:40.137423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.137391    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"]
Oct 30 18:14:40.140845 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.140829    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"
Oct 30 18:14:40.159692 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.159665    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"]
Oct 30 18:14:40.248872 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.248835    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sflqg\" (UniqueName: \"kubernetes.io/projected/6e4a363b-d216-4590-9324-04f1d31512e8-kube-api-access-sflqg\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd\" (UID: \"6e4a363b-d216-4590-9324-04f1d31512e8\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"
Oct 30 18:14:40.349219 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.349189    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sflqg\" (UniqueName: \"kubernetes.io/projected/6e4a363b-d216-4590-9324-04f1d31512e8-kube-api-access-sflqg\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd\" (UID: \"6e4a363b-d216-4590-9324-04f1d31512e8\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"
Oct 30 18:14:40.372231 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.372199    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sflqg\" (UniqueName: \"kubernetes.io/projected/6e4a363b-d216-4590-9324-04f1d31512e8-kube-api-access-sflqg\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd\" (UID: \"6e4a363b-d216-4590-9324-04f1d31512e8\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"
Oct 30 18:14:40.452171 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.452106    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"
Oct 30 18:14:40.550947 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.550900    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:40.578522 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.578494    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:40.651782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.651757    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:40.651956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.651926    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:40.666040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.666012    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:40.744816 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.744784    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"]
Oct 30 18:14:40.749001 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:14:40.748975    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e4a363b_d216_4590_9324_04f1d31512e8.slice/crio-4c46fbd71f3d30ea3aae496552683bdccd8474c6b711662d416c1a1330ce83df WatchSource:0}: Error finding container 4c46fbd71f3d30ea3aae496552683bdccd8474c6b711662d416c1a1330ce83df: Status 404 returned error can't find the container with id 4c46fbd71f3d30ea3aae496552683bdccd8474c6b711662d416c1a1330ce83df
Oct 30 18:14:40.768963 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.768938    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:40.798303 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.798279    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:40.969790 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:40.969758    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:41.266586 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.266555    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:14:41.270362 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:14:41.270337    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7763c17_8cc7_47c6_bf3f_16a7fa146631.slice/crio-b9c72f55e9f221765b73cda5726a8bde9cff862a937491d3820b5e02d5a04df9 WatchSource:0}: Error finding container b9c72f55e9f221765b73cda5726a8bde9cff862a937491d3820b5e02d5a04df9: Status 404 returned error can't find the container with id b9c72f55e9f221765b73cda5726a8bde9cff862a937491d3820b5e02d5a04df9
Oct 30 18:14:41.666689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.666644    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd" event={"ID":"6e4a363b-d216-4590-9324-04f1d31512e8","Type":"ContainerStarted","Data":"a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c"}
Oct 30 18:14:41.666933 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.666713    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd" event={"ID":"6e4a363b-d216-4590-9324-04f1d31512e8","Type":"ContainerStarted","Data":"4c46fbd71f3d30ea3aae496552683bdccd8474c6b711662d416c1a1330ce83df"}
Oct 30 18:14:41.668230 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.668209    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"f7763c17-8cc7-47c6-bf3f-16a7fa146631","Type":"ContainerStarted","Data":"dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8"}
Oct 30 18:14:41.668317 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.668238    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"f7763c17-8cc7-47c6-bf3f-16a7fa146631","Type":"ContainerStarted","Data":"b9c72f55e9f221765b73cda5726a8bde9cff862a937491d3820b5e02d5a04df9"}
Oct 30 18:14:41.668366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.668341    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:14:41.705946 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.705715    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd" podStartSLOduration=1.705697383 podStartE2EDuration="1.705697383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:14:41.705068429 +0000 UTC m=+1051.871886576" watchObservedRunningTime="2026-10-30 18:14:41.705697383 +0000 UTC m=+1051.872515510"
Oct 30 18:14:41.733422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.733390    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"]
Oct 30 18:14:41.733601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.733574    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl" podUID="05ae3ee5-62a1-4e54-9d18-62d1feabec32" containerName="thanos-querier" containerID="cri-o://fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242" gracePeriod=30
Oct 30 18:14:41.745990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.745943    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=9.745927416 podStartE2EDuration="9.745927416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:14:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:14:41.745181463 +0000 UTC m=+1051.911999588" watchObservedRunningTime="2026-10-30 18:14:41.745927416 +0000 UTC m=+1051.912745534"
Oct 30 18:14:41.986570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:41.986540    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"
Oct 30 18:14:42.064218 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.064177    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05ae3ee5-62a1-4e54-9d18-62d1feabec32-kube-api-access-b2trm\" (UniqueName: \"kubernetes.io/projected/05ae3ee5-62a1-4e54-9d18-62d1feabec32-kube-api-access-b2trm\") pod \"05ae3ee5-62a1-4e54-9d18-62d1feabec32\" (UID: \"05ae3ee5-62a1-4e54-9d18-62d1feabec32\") "
Oct 30 18:14:42.066681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.066639    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ae3ee5-62a1-4e54-9d18-62d1feabec32-kube-api-access-b2trm" pod "05ae3ee5-62a1-4e54-9d18-62d1feabec32" (UID: "05ae3ee5-62a1-4e54-9d18-62d1feabec32"). InnerVolumeSpecName "kube-api-access-b2trm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:14:42.165835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.165793    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2trm\" (UniqueName: \"kubernetes.io/projected/05ae3ee5-62a1-4e54-9d18-62d1feabec32-kube-api-access-b2trm\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:14:42.673625 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.673585    5746 generic.go:356] "Generic (PLEG): container finished" podID="05ae3ee5-62a1-4e54-9d18-62d1feabec32" containerID="fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242" exitCode=0
Oct 30 18:14:42.674071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.673666    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"
Oct 30 18:14:42.674071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.673673    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl" event={"ID":"05ae3ee5-62a1-4e54-9d18-62d1feabec32","Type":"ContainerDied","Data":"fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242"}
Oct 30 18:14:42.674071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.673723    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl" event={"ID":"05ae3ee5-62a1-4e54-9d18-62d1feabec32","Type":"ContainerDied","Data":"7f083d4799012e41b78b4c41cb83fc12b313207ed67dd0914c9c4c76707c8cf5"}
Oct 30 18:14:42.674071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.673747    5746 scope.go:122] "RemoveContainer" containerID="fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242"
Oct 30 18:14:42.682495 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.682472    5746 scope.go:122] "RemoveContainer" containerID="fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242"
Oct 30 18:14:42.682782 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:14:42.682741    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242\": container with ID starting with fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242 not found: ID does not exist" containerID="fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242"
Oct 30 18:14:42.682845 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.682791    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242"} err="failed to get container status \"fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242\": rpc error: code = NotFound desc = could not find container \"fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242\": container with ID starting with fccb99c4a2e61c7614294684fac1eeb99c41af50b62212b4f6b89d484a8bf242 not found: ID does not exist"
Oct 30 18:14:42.722968 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.722936    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"]
Oct 30 18:14:42.726766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:42.726739    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-mnkpl"]
Oct 30 18:14:42.793500 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:14:42.793461    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:14:44.008031 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:44.007998    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05ae3ee5-62a1-4e54-9d18-62d1feabec32" path="/var/lib/kubelet/pods/05ae3ee5-62a1-4e54-9d18-62d1feabec32/volumes"
Oct 30 18:14:52.676886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:14:52.676854    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:12.790333 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:15:12.790276    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:15:21.369569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.369473    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"]
Oct 30 18:15:21.370197 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.369700    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd" podUID="6e4a363b-d216-4590-9324-04f1d31512e8" containerName="thanos-querier" containerID="cri-o://a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c" gracePeriod=30
Oct 30 18:15:21.422677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.422644    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"]
Oct 30 18:15:21.423040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.423024    5746 cpu_manager.go:414] "containerMap: removing container" podUID="05ae3ee5-62a1-4e54-9d18-62d1feabec32" containerName="thanos-querier"
Oct 30 18:15:21.423104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.423042    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05ae3ee5-62a1-4e54-9d18-62d1feabec32" containerName="thanos-querier"
Oct 30 18:15:21.423104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.423097    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="05ae3ee5-62a1-4e54-9d18-62d1feabec32" containerName="thanos-querier"
Oct 30 18:15:21.426009 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.425988    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"
Oct 30 18:15:21.446028 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.445977    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"]
Oct 30 18:15:21.463388 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.463358    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"]
Oct 30 18:15:21.463599 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:15:21.463578    5746 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-r8rjc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8" podUID="abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a"
Oct 30 18:15:21.477297 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.477244    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8rjc\" (UniqueName: \"kubernetes.io/projected/abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a-kube-api-access-r8rjc\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8\" (UID: \"abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"
Oct 30 18:15:21.578501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.578471    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8rjc\" (UniqueName: \"kubernetes.io/projected/abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a-kube-api-access-r8rjc\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8\" (UID: \"abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"
Oct 30 18:15:21.604624 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.604587    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8rjc\" (UniqueName: \"kubernetes.io/projected/abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a-kube-api-access-r8rjc\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8\" (UID: \"abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"
Oct 30 18:15:21.625753 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.625691    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"
Oct 30 18:15:21.679665 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.679633    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e4a363b-d216-4590-9324-04f1d31512e8-kube-api-access-sflqg\" (UniqueName: \"kubernetes.io/projected/6e4a363b-d216-4590-9324-04f1d31512e8-kube-api-access-sflqg\") pod \"6e4a363b-d216-4590-9324-04f1d31512e8\" (UID: \"6e4a363b-d216-4590-9324-04f1d31512e8\") "
Oct 30 18:15:21.681896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.681868    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4a363b-d216-4590-9324-04f1d31512e8-kube-api-access-sflqg" pod "6e4a363b-d216-4590-9324-04f1d31512e8" (UID: "6e4a363b-d216-4590-9324-04f1d31512e8"). InnerVolumeSpecName "kube-api-access-sflqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:15:21.780962 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.780931    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sflqg\" (UniqueName: \"kubernetes.io/projected/6e4a363b-d216-4590-9324-04f1d31512e8-kube-api-access-sflqg\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:21.828616 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.828577    5746 generic.go:356] "Generic (PLEG): container finished" podID="6e4a363b-d216-4590-9324-04f1d31512e8" containerID="a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c" exitCode=0
Oct 30 18:15:21.828759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.828616    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd" event={"ID":"6e4a363b-d216-4590-9324-04f1d31512e8","Type":"ContainerDied","Data":"a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c"}
Oct 30 18:15:21.828759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.828647    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"
Oct 30 18:15:21.828759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.828656    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd" event={"ID":"6e4a363b-d216-4590-9324-04f1d31512e8","Type":"ContainerDied","Data":"4c46fbd71f3d30ea3aae496552683bdccd8474c6b711662d416c1a1330ce83df"}
Oct 30 18:15:21.828759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.828676    5746 scope.go:122] "RemoveContainer" containerID="a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c"
Oct 30 18:15:21.828759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.828621    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"
Oct 30 18:15:21.836267 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.836242    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"
Oct 30 18:15:21.837673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.837656    5746 scope.go:122] "RemoveContainer" containerID="a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c"
Oct 30 18:15:21.837954 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:15:21.837934    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c\": container with ID starting with a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c not found: ID does not exist" containerID="a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c"
Oct 30 18:15:21.838005 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.837965    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c"} err="failed to get container status \"a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c\": rpc error: code = NotFound desc = could not find container \"a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c\": container with ID starting with a8e35027c43a0dafe5c79f61268c482c741a8c6aff0fcc53a5b73f430a1a420c not found: ID does not exist"
Oct 30 18:15:21.928179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.928094    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"]
Oct 30 18:15:21.933822 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.933794    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-4xstd"]
Oct 30 18:15:21.983270 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.983239    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a-kube-api-access-r8rjc\" (UniqueName: \"kubernetes.io/projected/abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a-kube-api-access-r8rjc\") pod \"abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a\" (UID: \"abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a\") "
Oct 30 18:15:21.985414 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:21.985391    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a-kube-api-access-r8rjc" pod "abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a" (UID: "abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a"). InnerVolumeSpecName "kube-api-access-r8rjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:15:22.008974 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.008940    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e4a363b-d216-4590-9324-04f1d31512e8" path="/var/lib/kubelet/pods/6e4a363b-d216-4590-9324-04f1d31512e8/volumes"
Oct 30 18:15:22.084090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.084063    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8rjc\" (UniqueName: \"kubernetes.io/projected/abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a-kube-api-access-r8rjc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:22.287814 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.287724    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"]
Oct 30 18:15:22.288165 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.287969    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6" podUID="c0e7639e-c293-42a4-b8d8-887e1ef1f21d" containerName="ta-container" containerID="cri-o://298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379" gracePeriod=30
Oct 30 18:15:22.377472 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.377440    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:15:22.377986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.377667    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="f7763c17-8cc7-47c6-bf3f-16a7fa146631" containerName="perses" containerID="cri-o://dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8" gracePeriod=30
Oct 30 18:15:22.648898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.648866    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:15:22.650966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.650942    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:22.791085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.790995    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7763c17-8cc7-47c6-bf3f-16a7fa146631-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7763c17-8cc7-47c6-bf3f-16a7fa146631-plugins\") pod \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") "
Oct 30 18:15:22.791085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.791043    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7763c17-8cc7-47c6-bf3f-16a7fa146631-kube-api-access-lmgng\" (UniqueName: \"kubernetes.io/projected/f7763c17-8cc7-47c6-bf3f-16a7fa146631-kube-api-access-lmgng\") pod \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") "
Oct 30 18:15:22.791085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.791085    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-kube-api-access-slcwd\" (UniqueName: \"kubernetes.io/projected/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-kube-api-access-slcwd\") pod \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") "
Oct 30 18:15:22.791386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.791123    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-ta-internal\" (UniqueName: \"kubernetes.io/configmap/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-ta-internal\") pod \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") "
Oct 30 18:15:22.791386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.791282    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") "
Oct 30 18:15:22.791386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.791327    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-data-science-collector-ta-server-cert\") pod \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\" (UID: \"c0e7639e-c293-42a4-b8d8-887e1ef1f21d\") "
Oct 30 18:15:22.791386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.791381    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7763c17-8cc7-47c6-bf3f-16a7fa146631-config\" (UniqueName: \"kubernetes.io/configmap/f7763c17-8cc7-47c6-bf3f-16a7fa146631-config\") pod \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\" (UID: \"f7763c17-8cc7-47c6-bf3f-16a7fa146631\") "
Oct 30 18:15:22.791573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.791548    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-ta-internal" pod "c0e7639e-c293-42a4-b8d8-887e1ef1f21d" (UID: "c0e7639e-c293-42a4-b8d8-887e1ef1f21d"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:15:22.791678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.791629    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:22.792064 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.792039    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7763c17-8cc7-47c6-bf3f-16a7fa146631-config" pod "f7763c17-8cc7-47c6-bf3f-16a7fa146631" (UID: "f7763c17-8cc7-47c6-bf3f-16a7fa146631"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:15:22.793931 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.793869    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7763c17-8cc7-47c6-bf3f-16a7fa146631-kube-api-access-lmgng" pod "f7763c17-8cc7-47c6-bf3f-16a7fa146631" (UID: "f7763c17-8cc7-47c6-bf3f-16a7fa146631"). InnerVolumeSpecName "kube-api-access-lmgng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:15:22.794017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.793975    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-data-science-collector-ta-server-cert" pod "c0e7639e-c293-42a4-b8d8-887e1ef1f21d" (UID: "c0e7639e-c293-42a4-b8d8-887e1ef1f21d"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:15:22.794017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.794004    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-kube-api-access-slcwd" pod "c0e7639e-c293-42a4-b8d8-887e1ef1f21d" (UID: "c0e7639e-c293-42a4-b8d8-887e1ef1f21d"). InnerVolumeSpecName "kube-api-access-slcwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:15:22.795423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.795404    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "f7763c17-8cc7-47c6-bf3f-16a7fa146631" (UID: "f7763c17-8cc7-47c6-bf3f-16a7fa146631"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:15:22.834514 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.834478    5746 generic.go:356] "Generic (PLEG): container finished" podID="f7763c17-8cc7-47c6-bf3f-16a7fa146631" containerID="dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8" exitCode=0
Oct 30 18:15:22.834700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.834536    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:22.834700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.834558    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"f7763c17-8cc7-47c6-bf3f-16a7fa146631","Type":"ContainerDied","Data":"dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8"}
Oct 30 18:15:22.834700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.834595    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"f7763c17-8cc7-47c6-bf3f-16a7fa146631","Type":"ContainerDied","Data":"b9c72f55e9f221765b73cda5726a8bde9cff862a937491d3820b5e02d5a04df9"}
Oct 30 18:15:22.834700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.834612    5746 scope.go:122] "RemoveContainer" containerID="dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8"
Oct 30 18:15:22.836030 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.836008    5746 generic.go:356] "Generic (PLEG): container finished" podID="c0e7639e-c293-42a4-b8d8-887e1ef1f21d" containerID="298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379" exitCode=0
Oct 30 18:15:22.836146 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.836050    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"
Oct 30 18:15:22.836146 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.836098    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6" event={"ID":"c0e7639e-c293-42a4-b8d8-887e1ef1f21d","Type":"ContainerDied","Data":"298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379"}
Oct 30 18:15:22.836146 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.836125    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6" event={"ID":"c0e7639e-c293-42a4-b8d8-887e1ef1f21d","Type":"ContainerDied","Data":"646e10c5a098ec82a6bb6151706c72484ea4902a34728056fb4512e422a5d3f7"}
Oct 30 18:15:22.837210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.837195    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"
Oct 30 18:15:22.845698 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.845680    5746 scope.go:122] "RemoveContainer" containerID="dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8"
Oct 30 18:15:22.846026 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:15:22.846006    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8\": container with ID starting with dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8 not found: ID does not exist" containerID="dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8"
Oct 30 18:15:22.846096 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.846035    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8"} err="failed to get container status \"dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8\": rpc error: code = NotFound desc = could not find container \"dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8\": container with ID starting with dfb7e25a6fdc07bfdda42b7ba37a19f45fcb01f128723e05ccb1888e2d231bd8 not found: ID does not exist"
Oct 30 18:15:22.846096 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.846054    5746 scope.go:122] "RemoveContainer" containerID="298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379"
Oct 30 18:15:22.854098 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.854079    5746 scope.go:122] "RemoveContainer" containerID="298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379"
Oct 30 18:15:22.854415 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:15:22.854398    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379\": container with ID starting with 298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379 not found: ID does not exist" containerID="298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379"
Oct 30 18:15:22.854458 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.854426    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379"} err="failed to get container status \"298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379\": rpc error: code = NotFound desc = could not find container \"298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379\": container with ID starting with 298cdff81aa1c07cd1ac72f005bebe07878854de3a6c05c14c39daf3ed5aa379 not found: ID does not exist"
Oct 30 18:15:22.892016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.891987    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7763c17-8cc7-47c6-bf3f-16a7fa146631-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:22.892016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.892013    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmgng\" (UniqueName: \"kubernetes.io/projected/f7763c17-8cc7-47c6-bf3f-16a7fa146631-kube-api-access-lmgng\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:22.892210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.892023    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slcwd\" (UniqueName: \"kubernetes.io/projected/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-kube-api-access-slcwd\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:22.892210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.892050    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:15:22.892210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.892061    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/c0e7639e-c293-42a4-b8d8-887e1ef1f21d-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:22.905809 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.905788    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:15:22.907386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.907361    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"]
Oct 30 18:15:22.912879 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.912855    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-8pdx8"]
Oct 30 18:15:22.944240 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.944168    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7763c17-8cc7-47c6-bf3f-16a7fa146631-plugins" pod "f7763c17-8cc7-47c6-bf3f-16a7fa146631" (UID: "f7763c17-8cc7-47c6-bf3f-16a7fa146631"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:15:22.945296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.945272    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"]
Oct 30 18:15:22.949351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.949332    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-dd8cf967-ktkb6"]
Oct 30 18:15:22.993010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.992978    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7763c17-8cc7-47c6-bf3f-16a7fa146631-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:22.993010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:22.993005    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:15:23.177394 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:23.177360    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:15:23.182416 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:23.182391    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:15:24.007969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:24.007937    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a" path="/var/lib/kubelet/pods/abc6a73a-f61d-4226-b1a2-2ed1f8fb5e7a/volumes"
Oct 30 18:15:24.008358 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:24.008208    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0e7639e-c293-42a4-b8d8-887e1ef1f21d" path="/var/lib/kubelet/pods/c0e7639e-c293-42a4-b8d8-887e1ef1f21d/volumes"
Oct 30 18:15:24.008558 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:24.008545    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7763c17-8cc7-47c6-bf3f-16a7fa146631" path="/var/lib/kubelet/pods/f7763c17-8cc7-47c6-bf3f-16a7fa146631/volumes"
Oct 30 18:15:33.068354 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068317    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"]
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068617    5746 cpu_manager.go:414] "containerMap: removing container" podUID="f7763c17-8cc7-47c6-bf3f-16a7fa146631" containerName="perses"
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068631    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7763c17-8cc7-47c6-bf3f-16a7fa146631" containerName="perses"
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068641    5746 cpu_manager.go:414] "containerMap: removing container" podUID="6e4a363b-d216-4590-9324-04f1d31512e8" containerName="thanos-querier"
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068646    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e4a363b-d216-4590-9324-04f1d31512e8" containerName="thanos-querier"
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068665    5746 cpu_manager.go:414] "containerMap: removing container" podUID="c0e7639e-c293-42a4-b8d8-887e1ef1f21d" containerName="ta-container"
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068671    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0e7639e-c293-42a4-b8d8-887e1ef1f21d" containerName="ta-container"
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068722    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7763c17-8cc7-47c6-bf3f-16a7fa146631" containerName="perses"
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068732    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e4a363b-d216-4590-9324-04f1d31512e8" containerName="thanos-querier"
Oct 30 18:15:33.068984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.068738    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0e7639e-c293-42a4-b8d8-887e1ef1f21d" containerName="ta-container"
Oct 30 18:15:33.073111 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.073090    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"
Oct 30 18:15:33.091953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.091925    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"]
Oct 30 18:15:33.102987 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.102965    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:15:33.102987 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.102984    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:15:33.103121 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.102987    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:15:33.180640 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.180599    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cdl5\" (UniqueName: \"kubernetes.io/projected/d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d-kube-api-access-2cdl5\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2\" (UID: \"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"
Oct 30 18:15:33.281438 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.281402    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cdl5\" (UniqueName: \"kubernetes.io/projected/d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d-kube-api-access-2cdl5\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2\" (UID: \"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"
Oct 30 18:15:33.304837 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.304810    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cdl5\" (UniqueName: \"kubernetes.io/projected/d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d-kube-api-access-2cdl5\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2\" (UID: \"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"
Oct 30 18:15:33.384561 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.384529    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"
Oct 30 18:15:33.684559 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.684475    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"]
Oct 30 18:15:33.688040 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:15:33.688013    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9fc6d5c_09dc_4ebe_a3e5_9d109cda654d.slice/crio-ad64e86c09688fcd3239fb39cdf0ccb107be31fb8cb110407b0ec8ef6ae1a00e WatchSource:0}: Error finding container ad64e86c09688fcd3239fb39cdf0ccb107be31fb8cb110407b0ec8ef6ae1a00e: Status 404 returned error can't find the container with id ad64e86c09688fcd3239fb39cdf0ccb107be31fb8cb110407b0ec8ef6ae1a00e
Oct 30 18:15:33.880949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.880917    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2" event={"ID":"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d","Type":"ContainerStarted","Data":"688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8"}
Oct 30 18:15:33.880949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.880956    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2" event={"ID":"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d","Type":"ContainerStarted","Data":"ad64e86c09688fcd3239fb39cdf0ccb107be31fb8cb110407b0ec8ef6ae1a00e"}
Oct 30 18:15:33.920073 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:33.920010    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2" podStartSLOduration=0.919991041 podStartE2EDuration="919.991041ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:15:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:15:33.919273197 +0000 UTC m=+1104.086091324" watchObservedRunningTime="2026-10-30 18:15:33.919991041 +0000 UTC m=+1104.086809168"
Oct 30 18:15:34.097463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.097436    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:15:34.100711 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.100691    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.123035 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.122983    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:15:34.169212 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.169179    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:15:34.173564 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.173535    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"]
Oct 30 18:15:34.176783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.176768    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.187748 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.187726    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24njt\" (UniqueName: \"kubernetes.io/projected/874b8fa9-d753-4969-83ff-13ceba361446-kube-api-access-24njt\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.187848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.187763    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/874b8fa9-d753-4969-83ff-13ceba361446-config\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.187848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.187791    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/874b8fa9-d753-4969-83ff-13ceba361446-plugins\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.197500 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.197475    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"]
Oct 30 18:15:34.269379 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.269353    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:15:34.269796 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.269779    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:15:34.269862 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.269846    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-q6xnq\""
Oct 30 18:15:34.289220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.289192    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/874b8fa9-d753-4969-83ff-13ceba361446-plugins\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.289338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.289226    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbxwl\" (UniqueName: \"kubernetes.io/projected/60f26953-cd08-40ab-9b9d-890429236071-kube-api-access-zbxwl\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.289338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.289284    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24njt\" (UniqueName: \"kubernetes.io/projected/874b8fa9-d753-4969-83ff-13ceba361446-kube-api-access-24njt\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.289338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.289307    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/60f26953-cd08-40ab-9b9d-890429236071-ta-internal\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.289453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.289335    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60f26953-cd08-40ab-9b9d-890429236071-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.289453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.289402    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/874b8fa9-d753-4969-83ff-13ceba361446-config\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.289584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.289568    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/874b8fa9-d753-4969-83ff-13ceba361446-plugins\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.289953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.289936    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/874b8fa9-d753-4969-83ff-13ceba361446-config\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.312134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.312107    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24njt\" (UniqueName: \"kubernetes.io/projected/874b8fa9-d753-4969-83ff-13ceba361446-kube-api-access-24njt\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:34.390347 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.390272    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/60f26953-cd08-40ab-9b9d-890429236071-ta-internal\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.390347 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.390309    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60f26953-cd08-40ab-9b9d-890429236071-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.390536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.390368    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbxwl\" (UniqueName: \"kubernetes.io/projected/60f26953-cd08-40ab-9b9d-890429236071-kube-api-access-zbxwl\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.391062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.391003    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/60f26953-cd08-40ab-9b9d-890429236071-ta-internal\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.392875 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.392851    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60f26953-cd08-40ab-9b9d-890429236071-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.413065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.413039    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbxwl\" (UniqueName: \"kubernetes.io/projected/60f26953-cd08-40ab-9b9d-890429236071-kube-api-access-zbxwl\") pod \"data-science-collector-targetallocator-7c7ddc5b49-vhbh6\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.488643 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.488609    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.785128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.785050    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"]
Oct 30 18:15:34.788743 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:15:34.788717    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60f26953_cd08_40ab_9b9d_890429236071.slice/crio-3541676d735487ec2990f2d66dd15ee2d83d0669da63571b0d8b1b4af9c2bc61 WatchSource:0}: Error finding container 3541676d735487ec2990f2d66dd15ee2d83d0669da63571b0d8b1b4af9c2bc61: Status 404 returned error can't find the container with id 3541676d735487ec2990f2d66dd15ee2d83d0669da63571b0d8b1b4af9c2bc61
Oct 30 18:15:34.790877 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.790844    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:15:34.790981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.790899    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:15:34.886825 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.886783    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6" event={"ID":"60f26953-cd08-40ab-9b9d-890429236071","Type":"ContainerStarted","Data":"c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077"}
Oct 30 18:15:34.887012 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.886835    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6" event={"ID":"60f26953-cd08-40ab-9b9d-890429236071","Type":"ContainerStarted","Data":"3541676d735487ec2990f2d66dd15ee2d83d0669da63571b0d8b1b4af9c2bc61"}
Oct 30 18:15:34.887012 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.886999    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:34.887797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.887777    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6" podUID="60f26953-cd08-40ab-9b9d-890429236071" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.32:8080/readyz\": dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:15:34.928274 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:34.928226    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6" podStartSLOduration=0.928210858 podStartE2EDuration="928.210858ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:15:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:15:34.927370427 +0000 UTC m=+1105.094188556" watchObservedRunningTime="2026-10-30 18:15:34.928210858 +0000 UTC m=+1105.095028986"
Oct 30 18:15:35.891940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:35.891881    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:15:41.649255 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:41.649222    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:41.677074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:41.677041    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:41.750162 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:41.750124    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:41.750320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:41.750259    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:41.764393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:41.764363    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:41.867437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:41.867397    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:41.896676 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:41.896651    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:41.912256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:41.912193    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:42.209414 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.209337    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:15:42.212882 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:15:42.212851    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod874b8fa9_d753_4969_83ff_13ceba361446.slice/crio-0cccfaa938a93217edcd36607a61b851691a70c547a50e6ebbe03e769d75a940 WatchSource:0}: Error finding container 0cccfaa938a93217edcd36607a61b851691a70c547a50e6ebbe03e769d75a940: Status 404 returned error can't find the container with id 0cccfaa938a93217edcd36607a61b851691a70c547a50e6ebbe03e769d75a940
Oct 30 18:15:42.439648 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.439216    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"]
Oct 30 18:15:42.439648 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.439408    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2" podUID="d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d" containerName="thanos-querier" containerID="cri-o://688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8" gracePeriod=30
Oct 30 18:15:42.689826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.689799    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"
Oct 30 18:15:42.759438 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.759404    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d-kube-api-access-2cdl5\" (UniqueName: \"kubernetes.io/projected/d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d-kube-api-access-2cdl5\") pod \"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d\" (UID: \"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d\") "
Oct 30 18:15:42.761613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.761579    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d-kube-api-access-2cdl5" pod "d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d" (UID: "d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d"). InnerVolumeSpecName "kube-api-access-2cdl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:15:42.792863 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:15:42.792826    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:15:42.860454 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.860422    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cdl5\" (UniqueName: \"kubernetes.io/projected/d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d-kube-api-access-2cdl5\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:15:42.919795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.919763    5746 generic.go:356] "Generic (PLEG): container finished" podID="d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d" containerID="688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8" exitCode=0
Oct 30 18:15:42.920010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.919812    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"
Oct 30 18:15:42.920010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.919801    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2" event={"ID":"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d","Type":"ContainerDied","Data":"688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8"}
Oct 30 18:15:42.920010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.919949    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2" event={"ID":"d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d","Type":"ContainerDied","Data":"ad64e86c09688fcd3239fb39cdf0ccb107be31fb8cb110407b0ec8ef6ae1a00e"}
Oct 30 18:15:42.920010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.919980    5746 scope.go:122] "RemoveContainer" containerID="688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8"
Oct 30 18:15:42.921582 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.921518    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"874b8fa9-d753-4969-83ff-13ceba361446","Type":"ContainerStarted","Data":"19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891"}
Oct 30 18:15:42.921582 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.921554    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"874b8fa9-d753-4969-83ff-13ceba361446","Type":"ContainerStarted","Data":"0cccfaa938a93217edcd36607a61b851691a70c547a50e6ebbe03e769d75a940"}
Oct 30 18:15:42.921775 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.921662    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:15:42.929951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.929934    5746 scope.go:122] "RemoveContainer" containerID="688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8"
Oct 30 18:15:42.930232 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:15:42.930215    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8\": container with ID starting with 688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8 not found: ID does not exist" containerID="688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8"
Oct 30 18:15:42.930305 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.930238    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8"} err="failed to get container status \"688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8\": rpc error: code = NotFound desc = could not find container \"688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8\": container with ID starting with 688bd406c960583fae33736930ac380f64f31001524de52c104252ff4a029ae8 not found: ID does not exist"
Oct 30 18:15:42.970002 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.969972    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"]
Oct 30 18:15:42.974135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:42.974109    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-6kxw2"]
Oct 30 18:15:43.012846 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:43.012796    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=9.012782659 podStartE2EDuration="9.012782659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:15:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:15:43.012175404 +0000 UTC m=+1113.178993552" watchObservedRunningTime="2026-10-30 18:15:43.012782659 +0000 UTC m=+1113.179600782"
Oct 30 18:15:44.009417 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:44.009378    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d" path="/var/lib/kubelet/pods/d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d/volumes"
Oct 30 18:15:53.934012 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:15:53.933969    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:16:12.789298 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:16:12.789260    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:16:18.602746 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.602712    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"]
Oct 30 18:16:18.603251 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.603013    5746 cpu_manager.go:414] "containerMap: removing container" podUID="d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d" containerName="thanos-querier"
Oct 30 18:16:18.603251 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.603024    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d" containerName="thanos-querier"
Oct 30 18:16:18.603251 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.603075    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9fc6d5c-09dc-4ebe-a3e5-9d109cda654d" containerName="thanos-querier"
Oct 30 18:16:18.609118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.609097    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.639169 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.639138    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"]
Oct 30 18:16:18.640665 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.640645    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-ta-internal\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.640730 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.640681    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqjhf\" (UniqueName: \"kubernetes.io/projected/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-kube-api-access-bqjhf\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.640774 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.640753    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.741548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.741511    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.741713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.741587    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-ta-internal\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.741713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.741614    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqjhf\" (UniqueName: \"kubernetes.io/projected/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-kube-api-access-bqjhf\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.742407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.742335    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-ta-internal\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.744228 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.744207    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.764804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.764772    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqjhf\" (UniqueName: \"kubernetes.io/projected/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-kube-api-access-bqjhf\") pod \"data-science-collector-targetallocator-5cd47b6f44-h6rmh\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:18.920083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:18.920006    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:19.215960 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:19.215863    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"]
Oct 30 18:16:19.219141 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:16:19.219107    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d8c8896_f787_41e1_bcdd_6bb3a70dc8cc.slice/crio-c28bed4ba85a1aab61809367344cf752eebe69d8a50936100b8797ef78ca9063 WatchSource:0}: Error finding container c28bed4ba85a1aab61809367344cf752eebe69d8a50936100b8797ef78ca9063: Status 404 returned error can't find the container with id c28bed4ba85a1aab61809367344cf752eebe69d8a50936100b8797ef78ca9063
Oct 30 18:16:19.221650 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:19.221611    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:16:19.221747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:19.221678    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:16:20.068065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.068026    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh" event={"ID":"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc","Type":"ContainerStarted","Data":"96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4"}
Oct 30 18:16:20.068065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.068068    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh" event={"ID":"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc","Type":"ContainerStarted","Data":"c28bed4ba85a1aab61809367344cf752eebe69d8a50936100b8797ef78ca9063"}
Oct 30 18:16:20.068584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.068170    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:20.069763 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.069745    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:16:20.110159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.110115    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh" podStartSLOduration=2.110101197 podStartE2EDuration="2.110101197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:16:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:16:20.108979687 +0000 UTC m=+1150.275797816" watchObservedRunningTime="2026-10-30 18:16:20.110101197 +0000 UTC m=+1150.276919323"
Oct 30 18:16:20.180999 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.180962    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"]
Oct 30 18:16:20.181224 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.181186    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6" podUID="60f26953-cd08-40ab-9b9d-890429236071" containerName="ta-container" containerID="cri-o://c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077" gracePeriod=30
Oct 30 18:16:20.417798 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.417774    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:16:20.455123 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.455099    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60f26953-cd08-40ab-9b9d-890429236071-kube-api-access-zbxwl\" (UniqueName: \"kubernetes.io/projected/60f26953-cd08-40ab-9b9d-890429236071-kube-api-access-zbxwl\") pod \"60f26953-cd08-40ab-9b9d-890429236071\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") "
Oct 30 18:16:20.455275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.455200    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60f26953-cd08-40ab-9b9d-890429236071-ta-internal\" (UniqueName: \"kubernetes.io/configmap/60f26953-cd08-40ab-9b9d-890429236071-ta-internal\") pod \"60f26953-cd08-40ab-9b9d-890429236071\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") "
Oct 30 18:16:20.455275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.455243    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60f26953-cd08-40ab-9b9d-890429236071-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60f26953-cd08-40ab-9b9d-890429236071-data-science-collector-ta-server-cert\") pod \"60f26953-cd08-40ab-9b9d-890429236071\" (UID: \"60f26953-cd08-40ab-9b9d-890429236071\") "
Oct 30 18:16:20.455560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.455538    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60f26953-cd08-40ab-9b9d-890429236071-ta-internal" pod "60f26953-cd08-40ab-9b9d-890429236071" (UID: "60f26953-cd08-40ab-9b9d-890429236071"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:16:20.457312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.457282    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f26953-cd08-40ab-9b9d-890429236071-kube-api-access-zbxwl" pod "60f26953-cd08-40ab-9b9d-890429236071" (UID: "60f26953-cd08-40ab-9b9d-890429236071"). InnerVolumeSpecName "kube-api-access-zbxwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:16:20.457404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.457358    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f26953-cd08-40ab-9b9d-890429236071-data-science-collector-ta-server-cert" pod "60f26953-cd08-40ab-9b9d-890429236071" (UID: "60f26953-cd08-40ab-9b9d-890429236071"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:16:20.556626 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.556595    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/60f26953-cd08-40ab-9b9d-890429236071-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:16:20.556626 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.556622    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60f26953-cd08-40ab-9b9d-890429236071-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:16:20.556626 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:20.556632    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbxwl\" (UniqueName: \"kubernetes.io/projected/60f26953-cd08-40ab-9b9d-890429236071-kube-api-access-zbxwl\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:16:21.073351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.073317    5746 generic.go:356] "Generic (PLEG): container finished" podID="60f26953-cd08-40ab-9b9d-890429236071" containerID="c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077" exitCode=0
Oct 30 18:16:21.073828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.073368    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"
Oct 30 18:16:21.073828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.073413    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6" event={"ID":"60f26953-cd08-40ab-9b9d-890429236071","Type":"ContainerDied","Data":"c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077"}
Oct 30 18:16:21.073828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.073451    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6" event={"ID":"60f26953-cd08-40ab-9b9d-890429236071","Type":"ContainerDied","Data":"3541676d735487ec2990f2d66dd15ee2d83d0669da63571b0d8b1b4af9c2bc61"}
Oct 30 18:16:21.073828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.073471    5746 scope.go:122] "RemoveContainer" containerID="c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077"
Oct 30 18:16:21.082083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.082061    5746 scope.go:122] "RemoveContainer" containerID="c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077"
Oct 30 18:16:21.082348 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:16:21.082329    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077\": container with ID starting with c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077 not found: ID does not exist" containerID="c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077"
Oct 30 18:16:21.082418 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.082360    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077"} err="failed to get container status \"c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077\": rpc error: code = NotFound desc = could not find container \"c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077\": container with ID starting with c54bea10856f79dd17d2b60025ca58c83009c9d1f58af5486a9fae19ddc38077 not found: ID does not exist"
Oct 30 18:16:21.121818 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.121796    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"]
Oct 30 18:16:21.133406 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:21.133387    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c7ddc5b49-vhbh6"]
Oct 30 18:16:22.007655 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:16:22.007621    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60f26953-cd08-40ab-9b9d-890429236071" path="/var/lib/kubelet/pods/60f26953-cd08-40ab-9b9d-890429236071/volumes"
Oct 30 18:16:42.789578 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:16:42.789545    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:17:12.789411 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:17:12.789374    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:17:32.127542 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:32.127509    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"]
Oct 30 18:17:32.127966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:32.127785    5746 cpu_manager.go:414] "containerMap: removing container" podUID="60f26953-cd08-40ab-9b9d-890429236071" containerName="ta-container"
Oct 30 18:17:32.127966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:32.127795    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60f26953-cd08-40ab-9b9d-890429236071" containerName="ta-container"
Oct 30 18:17:32.127966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:32.127842    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="60f26953-cd08-40ab-9b9d-890429236071" containerName="ta-container"
Oct 30 18:17:32.130999 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:32.130983    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"
Oct 30 18:17:32.141150 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:32.141131    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"
Oct 30 18:17:32.147011 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:32.146980    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"]
Oct 30 18:17:32.433458 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:32.433379    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"]
Oct 30 18:17:32.436795 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:17:32.436764    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaac32c5e_8962_4ffc_a913_6e122874908d.slice/crio-52fb8e6e45fb5fb9aa067ec0c677211e3f2a417c39378cdc3952a0533c7db4b9 WatchSource:0}: Error finding container 52fb8e6e45fb5fb9aa067ec0c677211e3f2a417c39378cdc3952a0533c7db4b9: Status 404 returned error can't find the container with id 52fb8e6e45fb5fb9aa067ec0c677211e3f2a417c39378cdc3952a0533c7db4b9
Oct 30 18:17:33.353016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:33.352991    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b" event={"ID":"aac32c5e-8962-4ffc-a913-6e122874908d","Type":"ContainerStarted","Data":"ad664bdf53ee4746f659880094b166a39a2ee86c4433b7142372cae23b979cee"}
Oct 30 18:17:33.353349 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:33.353031    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b" event={"ID":"aac32c5e-8962-4ffc-a913-6e122874908d","Type":"ContainerStarted","Data":"52fb8e6e45fb5fb9aa067ec0c677211e3f2a417c39378cdc3952a0533c7db4b9"}
Oct 30 18:17:33.390569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:33.390525    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b" podStartSLOduration=0.586058572 podStartE2EDuration="1.390512278s" totalImagesPullingTime="804.453706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:17:33.389853427 +0000 UTC m=+1223.556671567" watchObservedRunningTime="2026-10-30 18:17:33.390512278 +0000 UTC m=+1223.557330405"
Oct 30 18:17:38.374082 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:38.374044    5746 generic.go:356] "Generic (PLEG): container finished" podID="aac32c5e-8962-4ffc-a913-6e122874908d" containerID="ad664bdf53ee4746f659880094b166a39a2ee86c4433b7142372cae23b979cee" exitCode=0
Oct 30 18:17:38.374434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:38.374102    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b" event={"ID":"aac32c5e-8962-4ffc-a913-6e122874908d","Type":"ContainerDied","Data":"ad664bdf53ee4746f659880094b166a39a2ee86c4433b7142372cae23b979cee"}
Oct 30 18:17:39.514104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:39.514077    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"
Oct 30 18:17:40.218579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:40.218544    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"]
Oct 30 18:17:40.220856 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:40.220826    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"]
Oct 30 18:17:40.383245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:40.383216    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"
Oct 30 18:17:40.383245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:40.383237    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52fb8e6e45fb5fb9aa067ec0c677211e3f2a417c39378cdc3952a0533c7db4b9"
Oct 30 18:17:40.398725 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:17:40.398692    5746 status_manager.go:1045] "Failed to get status for pod" err="pods \"korrel8r-ingress-18e36179bac4564b\" is forbidden: User \"system:node:ip-10-0-131-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"redhat-ods-monitoring\": no relationship found between node 'ip-10-0-131-31.ec2.internal' and this object" podUID="aac32c5e-8962-4ffc-a913-6e122874908d" pod="redhat-ods-monitoring/korrel8r-ingress-18e36179bac4564b"
Oct 30 18:17:42.007650 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:42.007611    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aac32c5e-8962-4ffc-a913-6e122874908d" path="/var/lib/kubelet/pods/aac32c5e-8962-4ffc-a913-6e122874908d/volumes"
Oct 30 18:17:42.789302 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:17:42.789269    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:17:49.728411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.728373    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"]
Oct 30 18:17:49.728791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.728659    5746 cpu_manager.go:414] "containerMap: removing container" podUID="aac32c5e-8962-4ffc-a913-6e122874908d" containerName="curl"
Oct 30 18:17:49.728791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.728669    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aac32c5e-8962-4ffc-a913-6e122874908d" containerName="curl"
Oct 30 18:17:49.728791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.728736    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="aac32c5e-8962-4ffc-a913-6e122874908d" containerName="curl"
Oct 30 18:17:49.731718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.731701    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.762272 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.762249    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"]
Oct 30 18:17:49.863066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.863033    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/42c6bad5-e306-4242-a7ff-7948cb27eb7a-ta-internal\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.863226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.863073    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/42c6bad5-e306-4242-a7ff-7948cb27eb7a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.863226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.863150    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhfv8\" (UniqueName: \"kubernetes.io/projected/42c6bad5-e306-4242-a7ff-7948cb27eb7a-kube-api-access-vhfv8\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.963735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.963689    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/42c6bad5-e306-4242-a7ff-7948cb27eb7a-ta-internal\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.963735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.963737    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/42c6bad5-e306-4242-a7ff-7948cb27eb7a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.963971 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.963765    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhfv8\" (UniqueName: \"kubernetes.io/projected/42c6bad5-e306-4242-a7ff-7948cb27eb7a-kube-api-access-vhfv8\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.964356 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.964333    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/42c6bad5-e306-4242-a7ff-7948cb27eb7a-ta-internal\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.966457 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.966429    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/42c6bad5-e306-4242-a7ff-7948cb27eb7a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:49.986510 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:49.986461    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhfv8\" (UniqueName: \"kubernetes.io/projected/42c6bad5-e306-4242-a7ff-7948cb27eb7a-kube-api-access-vhfv8\") pod \"data-science-collector-targetallocator-768bcff84c-87bqj\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:50.043556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.043530    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:50.339731 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.339650    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"]
Oct 30 18:17:50.342813 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:17:50.342789    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42c6bad5_e306_4242_a7ff_7948cb27eb7a.slice/crio-a477fdc76298085bf343297e6c07f412024e8ceb092da71fc4d2fb3fe098f20a WatchSource:0}: Error finding container a477fdc76298085bf343297e6c07f412024e8ceb092da71fc4d2fb3fe098f20a: Status 404 returned error can't find the container with id a477fdc76298085bf343297e6c07f412024e8ceb092da71fc4d2fb3fe098f20a
Oct 30 18:17:50.344946 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.344892    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:17:50.345029 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.344967    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:17:50.423776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.423748    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj" event={"ID":"42c6bad5-e306-4242-a7ff-7948cb27eb7a","Type":"ContainerStarted","Data":"28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3"}
Oct 30 18:17:50.423945 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.423787    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj" event={"ID":"42c6bad5-e306-4242-a7ff-7948cb27eb7a","Type":"ContainerStarted","Data":"a477fdc76298085bf343297e6c07f412024e8ceb092da71fc4d2fb3fe098f20a"}
Oct 30 18:17:50.424011 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.423952    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:50.425330 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.425308    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj" podUID="42c6bad5-e306-4242-a7ff-7948cb27eb7a" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.35:8080/readyz\": dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 18:17:50.464625 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:50.464555    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj" podStartSLOduration=1.464536266 podStartE2EDuration="1.464536266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:17:50.46368451 +0000 UTC m=+1240.630502638" watchObservedRunningTime="2026-10-30 18:17:50.464536266 +0000 UTC m=+1240.631354393"
Oct 30 18:17:51.429283 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.429251    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:17:51.499866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.499839    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"]
Oct 30 18:17:51.500066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.500032    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh" podUID="2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" containerName="ta-container" containerID="cri-o://96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4" gracePeriod=30
Oct 30 18:17:51.739005 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.738979    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:17:51.882685 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.882657    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-kube-api-access-bqjhf\" (UniqueName: \"kubernetes.io/projected/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-kube-api-access-bqjhf\") pod \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") "
Oct 30 18:17:51.882872 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.882731    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-data-science-collector-ta-server-cert\") pod \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") "
Oct 30 18:17:51.882872 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.882764    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-ta-internal\" (UniqueName: \"kubernetes.io/configmap/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-ta-internal\") pod \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\" (UID: \"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc\") "
Oct 30 18:17:51.883170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.883140    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-ta-internal" pod "2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" (UID: "2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:17:51.884889 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.884863    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-data-science-collector-ta-server-cert" pod "2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" (UID: "2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:17:51.884993 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.884925    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-kube-api-access-bqjhf" pod "2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" (UID: "2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc"). InnerVolumeSpecName "kube-api-access-bqjhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:17:51.983764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.983695    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:17:51.983764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.983720    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:17:51.983764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:51.983732    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqjhf\" (UniqueName: \"kubernetes.io/projected/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc-kube-api-access-bqjhf\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:17:52.433368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.433335    5746 generic.go:356] "Generic (PLEG): container finished" podID="2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" containerID="96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4" exitCode=0
Oct 30 18:17:52.433844 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.433379    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"
Oct 30 18:17:52.433844 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.433419    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh" event={"ID":"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc","Type":"ContainerDied","Data":"96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4"}
Oct 30 18:17:52.433844 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.433456    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh" event={"ID":"2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc","Type":"ContainerDied","Data":"c28bed4ba85a1aab61809367344cf752eebe69d8a50936100b8797ef78ca9063"}
Oct 30 18:17:52.433844 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.433471    5746 scope.go:122] "RemoveContainer" containerID="96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4"
Oct 30 18:17:52.441383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.441368    5746 scope.go:122] "RemoveContainer" containerID="96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4"
Oct 30 18:17:52.441674 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:17:52.441656    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4\": container with ID starting with 96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4 not found: ID does not exist" containerID="96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4"
Oct 30 18:17:52.441733 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.441682    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4"} err="failed to get container status \"96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4\": rpc error: code = NotFound desc = could not find container \"96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4\": container with ID starting with 96a402c5c6a42e129484a0b18cb741c3a5b259e237e15858b0a3856cc04990b4 not found: ID does not exist"
Oct 30 18:17:52.486462 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.486438    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"]
Oct 30 18:17:52.498258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:52.498233    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cd47b6f44-h6rmh"]
Oct 30 18:17:54.008165 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:17:54.008132    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" path="/var/lib/kubelet/pods/2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc/volumes"
Oct 30 18:18:07.064376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.064343    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"]
Oct 30 18:18:07.064878 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.064756    5746 cpu_manager.go:414] "containerMap: removing container" podUID="2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" containerName="ta-container"
Oct 30 18:18:07.064878 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.064774    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" containerName="ta-container"
Oct 30 18:18:07.064878 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.064846    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d8c8896-f787-41e1-bcdd-6bb3a70dc8cc" containerName="ta-container"
Oct 30 18:18:07.070074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.070054    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.088253 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.088230    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"]
Oct 30 18:18:07.209770 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.209745    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lmjw\" (UniqueName: \"kubernetes.io/projected/941087a5-5a0a-4f6b-b704-e44e4e58eafd-kube-api-access-7lmjw\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.209968 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.209786    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/941087a5-5a0a-4f6b-b704-e44e4e58eafd-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.209968 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.209943    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/941087a5-5a0a-4f6b-b704-e44e4e58eafd-ta-internal\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.311083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.311053    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/941087a5-5a0a-4f6b-b704-e44e4e58eafd-ta-internal\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.311192 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.311118    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lmjw\" (UniqueName: \"kubernetes.io/projected/941087a5-5a0a-4f6b-b704-e44e4e58eafd-kube-api-access-7lmjw\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.311192 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.311144    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/941087a5-5a0a-4f6b-b704-e44e4e58eafd-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.311672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.311651    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/941087a5-5a0a-4f6b-b704-e44e4e58eafd-ta-internal\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.313691 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.313673    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/941087a5-5a0a-4f6b-b704-e44e4e58eafd-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.334690 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.334632    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lmjw\" (UniqueName: \"kubernetes.io/projected/941087a5-5a0a-4f6b-b704-e44e4e58eafd-kube-api-access-7lmjw\") pod \"data-science-collector-targetallocator-5f54d8dc54-c728x\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.381451 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.381425    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:07.676726 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.676695    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"]
Oct 30 18:18:07.679802 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:18:07.679775    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod941087a5_5a0a_4f6b_b704_e44e4e58eafd.slice/crio-9b6bb68ce0124cd90286593512604e2e12b485e9f5c289d122a83babffd4ff07 WatchSource:0}: Error finding container 9b6bb68ce0124cd90286593512604e2e12b485e9f5c289d122a83babffd4ff07: Status 404 returned error can't find the container with id 9b6bb68ce0124cd90286593512604e2e12b485e9f5c289d122a83babffd4ff07
Oct 30 18:18:07.682191 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.682148    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:18:07.682269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:07.682226    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:18:08.495056 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:08.495022    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x" event={"ID":"941087a5-5a0a-4f6b-b704-e44e4e58eafd","Type":"ContainerStarted","Data":"b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5"}
Oct 30 18:18:08.495488 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:08.495064    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x" event={"ID":"941087a5-5a0a-4f6b-b704-e44e4e58eafd","Type":"ContainerStarted","Data":"9b6bb68ce0124cd90286593512604e2e12b485e9f5c289d122a83babffd4ff07"}
Oct 30 18:18:08.495488 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:08.495226    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:08.497276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:08.497247    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:08.535508 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:08.535460    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x" podStartSLOduration=1.535447367 podStartE2EDuration="1.535447367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:18:08.534663309 +0000 UTC m=+1258.701481436" watchObservedRunningTime="2026-10-30 18:18:08.535447367 +0000 UTC m=+1258.702265487"
Oct 30 18:18:08.605239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:08.605203    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"]
Oct 30 18:18:08.605465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:08.605439    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj" podUID="42c6bad5-e306-4242-a7ff-7948cb27eb7a" containerName="ta-container" containerID="cri-o://28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3" gracePeriod=30
Oct 30 18:18:08.853097 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:08.853074    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:18:09.026257 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.026173    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42c6bad5-e306-4242-a7ff-7948cb27eb7a-kube-api-access-vhfv8\" (UniqueName: \"kubernetes.io/projected/42c6bad5-e306-4242-a7ff-7948cb27eb7a-kube-api-access-vhfv8\") pod \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") "
Oct 30 18:18:09.026257 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.026249    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42c6bad5-e306-4242-a7ff-7948cb27eb7a-ta-internal\" (UniqueName: \"kubernetes.io/configmap/42c6bad5-e306-4242-a7ff-7948cb27eb7a-ta-internal\") pod \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") "
Oct 30 18:18:09.026483 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.026290    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42c6bad5-e306-4242-a7ff-7948cb27eb7a-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/42c6bad5-e306-4242-a7ff-7948cb27eb7a-data-science-collector-ta-server-cert\") pod \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\" (UID: \"42c6bad5-e306-4242-a7ff-7948cb27eb7a\") "
Oct 30 18:18:09.026598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.026572    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42c6bad5-e306-4242-a7ff-7948cb27eb7a-ta-internal" pod "42c6bad5-e306-4242-a7ff-7948cb27eb7a" (UID: "42c6bad5-e306-4242-a7ff-7948cb27eb7a"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:18:09.028353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.028329    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42c6bad5-e306-4242-a7ff-7948cb27eb7a-data-science-collector-ta-server-cert" pod "42c6bad5-e306-4242-a7ff-7948cb27eb7a" (UID: "42c6bad5-e306-4242-a7ff-7948cb27eb7a"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:18:09.028440 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.028412    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42c6bad5-e306-4242-a7ff-7948cb27eb7a-kube-api-access-vhfv8" pod "42c6bad5-e306-4242-a7ff-7948cb27eb7a" (UID: "42c6bad5-e306-4242-a7ff-7948cb27eb7a"). InnerVolumeSpecName "kube-api-access-vhfv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:18:09.127514 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.127479    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhfv8\" (UniqueName: \"kubernetes.io/projected/42c6bad5-e306-4242-a7ff-7948cb27eb7a-kube-api-access-vhfv8\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:09.127514 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.127507    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/42c6bad5-e306-4242-a7ff-7948cb27eb7a-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:09.127514 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.127518    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/42c6bad5-e306-4242-a7ff-7948cb27eb7a-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:09.500114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.500078    5746 generic.go:356] "Generic (PLEG): container finished" podID="42c6bad5-e306-4242-a7ff-7948cb27eb7a" containerID="28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3" exitCode=0
Oct 30 18:18:09.500572 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.500119    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"
Oct 30 18:18:09.500572 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.500166    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj" event={"ID":"42c6bad5-e306-4242-a7ff-7948cb27eb7a","Type":"ContainerDied","Data":"28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3"}
Oct 30 18:18:09.500572 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.500212    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj" event={"ID":"42c6bad5-e306-4242-a7ff-7948cb27eb7a","Type":"ContainerDied","Data":"a477fdc76298085bf343297e6c07f412024e8ceb092da71fc4d2fb3fe098f20a"}
Oct 30 18:18:09.500572 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.500229    5746 scope.go:122] "RemoveContainer" containerID="28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3"
Oct 30 18:18:09.508848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.508828    5746 scope.go:122] "RemoveContainer" containerID="28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3"
Oct 30 18:18:09.509201 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:18:09.509181    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3\": container with ID starting with 28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3 not found: ID does not exist" containerID="28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3"
Oct 30 18:18:09.509254 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.509211    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3"} err="failed to get container status \"28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3\": rpc error: code = NotFound desc = could not find container \"28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3\": container with ID starting with 28025ee306b1fd2d35d98cb7a86339fe11e0954e140be7426e42c51a4439ecf3 not found: ID does not exist"
Oct 30 18:18:09.548564 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.548536    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"]
Oct 30 18:18:09.560058 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:09.560005    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-768bcff84c-87bqj"]
Oct 30 18:18:10.009521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:10.009492    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42c6bad5-e306-4242-a7ff-7948cb27eb7a" path="/var/lib/kubelet/pods/42c6bad5-e306-4242-a7ff-7948cb27eb7a/volumes"
Oct 30 18:18:12.789652 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:18:12.789616    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:18:24.542570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.542484    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"]
Oct 30 18:18:24.543181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.542925    5746 cpu_manager.go:414] "containerMap: removing container" podUID="42c6bad5-e306-4242-a7ff-7948cb27eb7a" containerName="ta-container"
Oct 30 18:18:24.543181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.542943    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42c6bad5-e306-4242-a7ff-7948cb27eb7a" containerName="ta-container"
Oct 30 18:18:24.543181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.543016    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="42c6bad5-e306-4242-a7ff-7948cb27eb7a" containerName="ta-container"
Oct 30 18:18:24.547322 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.547301    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.574161 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.574137    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"]
Oct 30 18:18:24.656701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.656674    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cf73f956-dc36-4310-b1f0-2884e314712e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.656835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.656708    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvsvl\" (UniqueName: \"kubernetes.io/projected/cf73f956-dc36-4310-b1f0-2884e314712e-kube-api-access-wvsvl\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.656835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.656737    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/cf73f956-dc36-4310-b1f0-2884e314712e-ta-internal\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.757615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.757580    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cf73f956-dc36-4310-b1f0-2884e314712e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.757615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.757616    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvsvl\" (UniqueName: \"kubernetes.io/projected/cf73f956-dc36-4310-b1f0-2884e314712e-kube-api-access-wvsvl\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.757793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.757642    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/cf73f956-dc36-4310-b1f0-2884e314712e-ta-internal\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.758226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.758205    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/cf73f956-dc36-4310-b1f0-2884e314712e-ta-internal\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.760209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.760193    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cf73f956-dc36-4310-b1f0-2884e314712e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.780372 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.780343    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvsvl\" (UniqueName: \"kubernetes.io/projected/cf73f956-dc36-4310-b1f0-2884e314712e-kube-api-access-wvsvl\") pod \"data-science-collector-targetallocator-6948bb6854-zfgs8\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:24.858093 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:24.858069    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:25.177303 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:25.177228    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"]
Oct 30 18:18:25.180417 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:18:25.180389    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf73f956_dc36_4310_b1f0_2884e314712e.slice/crio-27a2d2f84b8120db942eb78865c6fb67fadf0f0d6fb7f1cdcf4c8d2eb05e26aa WatchSource:0}: Error finding container 27a2d2f84b8120db942eb78865c6fb67fadf0f0d6fb7f1cdcf4c8d2eb05e26aa: Status 404 returned error can't find the container with id 27a2d2f84b8120db942eb78865c6fb67fadf0f0d6fb7f1cdcf4c8d2eb05e26aa
Oct 30 18:18:25.182992 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:25.182950    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:18:25.183151 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:25.183025    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:18:25.567713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:25.567627    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8" event={"ID":"cf73f956-dc36-4310-b1f0-2884e314712e","Type":"ContainerStarted","Data":"25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3"}
Oct 30 18:18:25.567713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:25.567671    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8" event={"ID":"cf73f956-dc36-4310-b1f0-2884e314712e","Type":"ContainerStarted","Data":"27a2d2f84b8120db942eb78865c6fb67fadf0f0d6fb7f1cdcf4c8d2eb05e26aa"}
Oct 30 18:18:25.568224 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:25.567818    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:25.568988 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:25.568963    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8" podUID="cf73f956-dc36-4310-b1f0-2884e314712e" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.37:8080/readyz\": dial tcp 10.133.0.37:8080: connect: connection refused"
Oct 30 18:18:25.608692 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:25.608642    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8" podStartSLOduration=1.608629667 podStartE2EDuration="1.608629667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:18:25.608603406 +0000 UTC m=+1275.775421534" watchObservedRunningTime="2026-10-30 18:18:25.608629667 +0000 UTC m=+1275.775447791"
Oct 30 18:18:26.572831 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.572804    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:26.646253 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.646220    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"]
Oct 30 18:18:26.646464 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.646436    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x" podUID="941087a5-5a0a-4f6b-b704-e44e4e58eafd" containerName="ta-container" containerID="cri-o://b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5" gracePeriod=30
Oct 30 18:18:26.884826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.884802    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:26.979042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.979013    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/941087a5-5a0a-4f6b-b704-e44e4e58eafd-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/941087a5-5a0a-4f6b-b704-e44e4e58eafd-data-science-collector-ta-server-cert\") pod \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") "
Oct 30 18:18:26.979245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.979076    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/941087a5-5a0a-4f6b-b704-e44e4e58eafd-ta-internal\" (UniqueName: \"kubernetes.io/configmap/941087a5-5a0a-4f6b-b704-e44e4e58eafd-ta-internal\") pod \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") "
Oct 30 18:18:26.979245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.979100    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/941087a5-5a0a-4f6b-b704-e44e4e58eafd-kube-api-access-7lmjw\" (UniqueName: \"kubernetes.io/projected/941087a5-5a0a-4f6b-b704-e44e4e58eafd-kube-api-access-7lmjw\") pod \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\" (UID: \"941087a5-5a0a-4f6b-b704-e44e4e58eafd\") "
Oct 30 18:18:26.979487 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.979458    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/941087a5-5a0a-4f6b-b704-e44e4e58eafd-ta-internal" pod "941087a5-5a0a-4f6b-b704-e44e4e58eafd" (UID: "941087a5-5a0a-4f6b-b704-e44e4e58eafd"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:18:26.981385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.981362    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/941087a5-5a0a-4f6b-b704-e44e4e58eafd-data-science-collector-ta-server-cert" pod "941087a5-5a0a-4f6b-b704-e44e4e58eafd" (UID: "941087a5-5a0a-4f6b-b704-e44e4e58eafd"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:18:26.981461 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:26.981376    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/941087a5-5a0a-4f6b-b704-e44e4e58eafd-kube-api-access-7lmjw" pod "941087a5-5a0a-4f6b-b704-e44e4e58eafd" (UID: "941087a5-5a0a-4f6b-b704-e44e4e58eafd"). InnerVolumeSpecName "kube-api-access-7lmjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:18:27.080671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.080621    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/941087a5-5a0a-4f6b-b704-e44e4e58eafd-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:27.080671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.080671    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lmjw\" (UniqueName: \"kubernetes.io/projected/941087a5-5a0a-4f6b-b704-e44e4e58eafd-kube-api-access-7lmjw\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:27.080671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.080685    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/941087a5-5a0a-4f6b-b704-e44e4e58eafd-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:27.577271 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.577237    5746 generic.go:356] "Generic (PLEG): container finished" podID="941087a5-5a0a-4f6b-b704-e44e4e58eafd" containerID="b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5" exitCode=0
Oct 30 18:18:27.577869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.577287    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"
Oct 30 18:18:27.577869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.577317    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x" event={"ID":"941087a5-5a0a-4f6b-b704-e44e4e58eafd","Type":"ContainerDied","Data":"b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5"}
Oct 30 18:18:27.577869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.577354    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x" event={"ID":"941087a5-5a0a-4f6b-b704-e44e4e58eafd","Type":"ContainerDied","Data":"9b6bb68ce0124cd90286593512604e2e12b485e9f5c289d122a83babffd4ff07"}
Oct 30 18:18:27.577869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.577370    5746 scope.go:122] "RemoveContainer" containerID="b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5"
Oct 30 18:18:27.585698 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.585683    5746 scope.go:122] "RemoveContainer" containerID="b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5"
Oct 30 18:18:27.585979 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:18:27.585954    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5\": container with ID starting with b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5 not found: ID does not exist" containerID="b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5"
Oct 30 18:18:27.586080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.585984    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5"} err="failed to get container status \"b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5\": rpc error: code = NotFound desc = could not find container \"b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5\": container with ID starting with b7258570afdc41fb0c3655cf0cd97051213237707dfc2e42b5837b2bb08afea5 not found: ID does not exist"
Oct 30 18:18:27.625422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.625392    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"]
Oct 30 18:18:27.636978 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:27.636953    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f54d8dc54-c728x"]
Oct 30 18:18:28.008783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:28.008749    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="941087a5-5a0a-4f6b-b704-e44e4e58eafd" path="/var/lib/kubelet/pods/941087a5-5a0a-4f6b-b704-e44e4e58eafd/volumes"
Oct 30 18:18:41.915486 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.915452    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"]
Oct 30 18:18:41.916131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.915877    5746 cpu_manager.go:414] "containerMap: removing container" podUID="941087a5-5a0a-4f6b-b704-e44e4e58eafd" containerName="ta-container"
Oct 30 18:18:41.916131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.915896    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="941087a5-5a0a-4f6b-b704-e44e4e58eafd" containerName="ta-container"
Oct 30 18:18:41.916131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.916024    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="941087a5-5a0a-4f6b-b704-e44e4e58eafd" containerName="ta-container"
Oct 30 18:18:41.920340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.920318    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:41.947888 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.947850    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"]
Oct 30 18:18:41.995209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.995175    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fc946fb6-04a8-4f7e-81ec-724b2a63565c-ta-internal\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:41.995407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.995224    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9f7l\" (UniqueName: \"kubernetes.io/projected/fc946fb6-04a8-4f7e-81ec-724b2a63565c-kube-api-access-v9f7l\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:41.995407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:41.995397    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fc946fb6-04a8-4f7e-81ec-724b2a63565c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.096062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.096030    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fc946fb6-04a8-4f7e-81ec-724b2a63565c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.096232 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.096076    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fc946fb6-04a8-4f7e-81ec-724b2a63565c-ta-internal\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.096232 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.096112    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9f7l\" (UniqueName: \"kubernetes.io/projected/fc946fb6-04a8-4f7e-81ec-724b2a63565c-kube-api-access-v9f7l\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.096804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.096785    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fc946fb6-04a8-4f7e-81ec-724b2a63565c-ta-internal\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.098605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.098587    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fc946fb6-04a8-4f7e-81ec-724b2a63565c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.118821 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.118795    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9f7l\" (UniqueName: \"kubernetes.io/projected/fc946fb6-04a8-4f7e-81ec-724b2a63565c-kube-api-access-v9f7l\") pod \"data-science-collector-targetallocator-5d979c9fdc-6qw49\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.231612 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.231526    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.526458 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.526389    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"]
Oct 30 18:18:42.529321 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:18:42.529294    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc946fb6_04a8_4f7e_81ec_724b2a63565c.slice/crio-d051d6fa4f2bd6ae8fe0f6f06b781737f9957dd7102370e83d32308fc30b7bca WatchSource:0}: Error finding container d051d6fa4f2bd6ae8fe0f6f06b781737f9957dd7102370e83d32308fc30b7bca: Status 404 returned error can't find the container with id d051d6fa4f2bd6ae8fe0f6f06b781737f9957dd7102370e83d32308fc30b7bca
Oct 30 18:18:42.531536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.531496    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:18:42.531631 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.531573    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:18:42.640404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.640372    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49" event={"ID":"fc946fb6-04a8-4f7e-81ec-724b2a63565c","Type":"ContainerStarted","Data":"3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d"}
Oct 30 18:18:42.640588 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.640415    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49" event={"ID":"fc946fb6-04a8-4f7e-81ec-724b2a63565c","Type":"ContainerStarted","Data":"d051d6fa4f2bd6ae8fe0f6f06b781737f9957dd7102370e83d32308fc30b7bca"}
Oct 30 18:18:42.640588 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.640561    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:42.641512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.641490    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49" podUID="fc946fb6-04a8-4f7e-81ec-724b2a63565c" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.38:8080/readyz\": dial tcp 10.133.0.38:8080: connect: connection refused"
Oct 30 18:18:42.680646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:42.680584    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49" podStartSLOduration=1.680567285 podStartE2EDuration="1.680567285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:18:42.680109586 +0000 UTC m=+1292.846927713" watchObservedRunningTime="2026-10-30 18:18:42.680567285 +0000 UTC m=+1292.847385413"
Oct 30 18:18:42.790257 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:18:42.790177    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:18:43.645431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:43.645406    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:18:43.716360 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:43.716326    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"]
Oct 30 18:18:43.716531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:43.716511    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8" podUID="cf73f956-dc36-4310-b1f0-2884e314712e" containerName="ta-container" containerID="cri-o://25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3" gracePeriod=30
Oct 30 18:18:43.951856 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:43.951834    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:44.110287 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.110257    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf73f956-dc36-4310-b1f0-2884e314712e-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cf73f956-dc36-4310-b1f0-2884e314712e-data-science-collector-ta-server-cert\") pod \"cf73f956-dc36-4310-b1f0-2884e314712e\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") "
Oct 30 18:18:44.110460 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.110342    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf73f956-dc36-4310-b1f0-2884e314712e-kube-api-access-wvsvl\" (UniqueName: \"kubernetes.io/projected/cf73f956-dc36-4310-b1f0-2884e314712e-kube-api-access-wvsvl\") pod \"cf73f956-dc36-4310-b1f0-2884e314712e\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") "
Oct 30 18:18:44.110460 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.110394    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf73f956-dc36-4310-b1f0-2884e314712e-ta-internal\" (UniqueName: \"kubernetes.io/configmap/cf73f956-dc36-4310-b1f0-2884e314712e-ta-internal\") pod \"cf73f956-dc36-4310-b1f0-2884e314712e\" (UID: \"cf73f956-dc36-4310-b1f0-2884e314712e\") "
Oct 30 18:18:44.110775 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.110747    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf73f956-dc36-4310-b1f0-2884e314712e-ta-internal" pod "cf73f956-dc36-4310-b1f0-2884e314712e" (UID: "cf73f956-dc36-4310-b1f0-2884e314712e"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:18:44.112594 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.112573    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf73f956-dc36-4310-b1f0-2884e314712e-data-science-collector-ta-server-cert" pod "cf73f956-dc36-4310-b1f0-2884e314712e" (UID: "cf73f956-dc36-4310-b1f0-2884e314712e"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:18:44.112699 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.112682    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf73f956-dc36-4310-b1f0-2884e314712e-kube-api-access-wvsvl" pod "cf73f956-dc36-4310-b1f0-2884e314712e" (UID: "cf73f956-dc36-4310-b1f0-2884e314712e"). InnerVolumeSpecName "kube-api-access-wvsvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:18:44.211205 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.211130    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cf73f956-dc36-4310-b1f0-2884e314712e-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:44.211205 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.211157    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvsvl\" (UniqueName: \"kubernetes.io/projected/cf73f956-dc36-4310-b1f0-2884e314712e-kube-api-access-wvsvl\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:44.211205 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.211168    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/cf73f956-dc36-4310-b1f0-2884e314712e-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:18:44.649563 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.649518    5746 generic.go:356] "Generic (PLEG): container finished" podID="cf73f956-dc36-4310-b1f0-2884e314712e" containerID="25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3" exitCode=0
Oct 30 18:18:44.650004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.649586    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"
Oct 30 18:18:44.650004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.649602    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8" event={"ID":"cf73f956-dc36-4310-b1f0-2884e314712e","Type":"ContainerDied","Data":"25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3"}
Oct 30 18:18:44.650004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.649648    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8" event={"ID":"cf73f956-dc36-4310-b1f0-2884e314712e","Type":"ContainerDied","Data":"27a2d2f84b8120db942eb78865c6fb67fadf0f0d6fb7f1cdcf4c8d2eb05e26aa"}
Oct 30 18:18:44.650004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.649667    5746 scope.go:122] "RemoveContainer" containerID="25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3"
Oct 30 18:18:44.664501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.664482    5746 scope.go:122] "RemoveContainer" containerID="25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3"
Oct 30 18:18:44.664763 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:18:44.664742    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3\": container with ID starting with 25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3 not found: ID does not exist" containerID="25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3"
Oct 30 18:18:44.664828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.664770    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3"} err="failed to get container status \"25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3\": rpc error: code = NotFound desc = could not find container \"25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3\": container with ID starting with 25ef558bc2f6469946b995f583e3816c8b192bb480c130da3523f14f7839a9b3 not found: ID does not exist"
Oct 30 18:18:44.698573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.698517    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"]
Oct 30 18:18:44.709946 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:44.709896    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6948bb6854-zfgs8"]
Oct 30 18:18:46.008894 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:46.008860    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf73f956-dc36-4310-b1f0-2884e314712e" path="/var/lib/kubelet/pods/cf73f956-dc36-4310-b1f0-2884e314712e/volumes"
Oct 30 18:18:58.831178 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.831146    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"]
Oct 30 18:18:58.831662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.831533    5746 cpu_manager.go:414] "containerMap: removing container" podUID="cf73f956-dc36-4310-b1f0-2884e314712e" containerName="ta-container"
Oct 30 18:18:58.831662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.831550    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf73f956-dc36-4310-b1f0-2884e314712e" containerName="ta-container"
Oct 30 18:18:58.831662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.831645    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf73f956-dc36-4310-b1f0-2884e314712e" containerName="ta-container"
Oct 30 18:18:58.834395 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.834376    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:58.863125 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.863103    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"]
Oct 30 18:18:58.935596 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.935571    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:58.935754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.935603    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-ta-internal\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:58.935754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:58.935632    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lm7r\" (UniqueName: \"kubernetes.io/projected/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-kube-api-access-8lm7r\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.037117 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.037069    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lm7r\" (UniqueName: \"kubernetes.io/projected/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-kube-api-access-8lm7r\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.037292 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.037183    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.037292 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.037212    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-ta-internal\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.037984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.037952    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-ta-internal\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.039815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.039796    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.059576 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.059546    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lm7r\" (UniqueName: \"kubernetes.io/projected/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-kube-api-access-8lm7r\") pod \"data-science-collector-targetallocator-7f6dcd8b56-9vxr8\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.144672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.144643    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.438978 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.438875    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"]
Oct 30 18:18:59.442178 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:18:59.442150    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0074edeb_4e9b_45c6_89bc_ba91c3787ea1.slice/crio-a8a1edf60858b376e4949491f3ceaa83dc825d07dd7955558dee7e06cd97ec8f WatchSource:0}: Error finding container a8a1edf60858b376e4949491f3ceaa83dc825d07dd7955558dee7e06cd97ec8f: Status 404 returned error can't find the container with id a8a1edf60858b376e4949491f3ceaa83dc825d07dd7955558dee7e06cd97ec8f
Oct 30 18:18:59.444201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.444170    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:18:59.444377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.444230    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:18:59.710622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.710529    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8" event={"ID":"0074edeb-4e9b-45c6-89bc-ba91c3787ea1","Type":"ContainerStarted","Data":"5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74"}
Oct 30 18:18:59.710622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.710567    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8" event={"ID":"0074edeb-4e9b-45c6-89bc-ba91c3787ea1","Type":"ContainerStarted","Data":"a8a1edf60858b376e4949491f3ceaa83dc825d07dd7955558dee7e06cd97ec8f"}
Oct 30 18:18:59.710818 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.710710    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:18:59.711973 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.711949    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8" podUID="0074edeb-4e9b-45c6-89bc-ba91c3787ea1" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.39:8080/readyz\": dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:18:59.774499 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:18:59.774442    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8" podStartSLOduration=1.774423294 podStartE2EDuration="1.774423294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:18:59.773784627 +0000 UTC m=+1309.940602750" watchObservedRunningTime="2026-10-30 18:18:59.774423294 +0000 UTC m=+1309.941241422"
Oct 30 18:19:00.716289 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:00.716262    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:19:00.786199 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:00.786167    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"]
Oct 30 18:19:00.786388 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:00.786356    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49" podUID="fc946fb6-04a8-4f7e-81ec-724b2a63565c" containerName="ta-container" containerID="cri-o://3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d" gracePeriod=30
Oct 30 18:19:01.036852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.036829    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:19:01.155598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.155561    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc946fb6-04a8-4f7e-81ec-724b2a63565c-ta-internal\" (UniqueName: \"kubernetes.io/configmap/fc946fb6-04a8-4f7e-81ec-724b2a63565c-ta-internal\") pod \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") "
Oct 30 18:19:01.155598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.155600    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc946fb6-04a8-4f7e-81ec-724b2a63565c-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fc946fb6-04a8-4f7e-81ec-724b2a63565c-data-science-collector-ta-server-cert\") pod \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") "
Oct 30 18:19:01.155865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.155627    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc946fb6-04a8-4f7e-81ec-724b2a63565c-kube-api-access-v9f7l\" (UniqueName: \"kubernetes.io/projected/fc946fb6-04a8-4f7e-81ec-724b2a63565c-kube-api-access-v9f7l\") pod \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\" (UID: \"fc946fb6-04a8-4f7e-81ec-724b2a63565c\") "
Oct 30 18:19:01.156033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.156005    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc946fb6-04a8-4f7e-81ec-724b2a63565c-ta-internal" pod "fc946fb6-04a8-4f7e-81ec-724b2a63565c" (UID: "fc946fb6-04a8-4f7e-81ec-724b2a63565c"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:19:01.157929 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.157890    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc946fb6-04a8-4f7e-81ec-724b2a63565c-kube-api-access-v9f7l" pod "fc946fb6-04a8-4f7e-81ec-724b2a63565c" (UID: "fc946fb6-04a8-4f7e-81ec-724b2a63565c"). InnerVolumeSpecName "kube-api-access-v9f7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:19:01.158018 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.157895    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc946fb6-04a8-4f7e-81ec-724b2a63565c-data-science-collector-ta-server-cert" pod "fc946fb6-04a8-4f7e-81ec-724b2a63565c" (UID: "fc946fb6-04a8-4f7e-81ec-724b2a63565c"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:19:01.256509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.256474    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fc946fb6-04a8-4f7e-81ec-724b2a63565c-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:01.256509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.256506    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fc946fb6-04a8-4f7e-81ec-724b2a63565c-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:01.256509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.256517    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9f7l\" (UniqueName: \"kubernetes.io/projected/fc946fb6-04a8-4f7e-81ec-724b2a63565c-kube-api-access-v9f7l\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:01.720048 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.720007    5746 generic.go:356] "Generic (PLEG): container finished" podID="fc946fb6-04a8-4f7e-81ec-724b2a63565c" containerID="3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d" exitCode=0
Oct 30 18:19:01.720484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.720054    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"
Oct 30 18:19:01.720484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.720092    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49" event={"ID":"fc946fb6-04a8-4f7e-81ec-724b2a63565c","Type":"ContainerDied","Data":"3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d"}
Oct 30 18:19:01.720484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.720132    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49" event={"ID":"fc946fb6-04a8-4f7e-81ec-724b2a63565c","Type":"ContainerDied","Data":"d051d6fa4f2bd6ae8fe0f6f06b781737f9957dd7102370e83d32308fc30b7bca"}
Oct 30 18:19:01.720484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.720149    5746 scope.go:122] "RemoveContainer" containerID="3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d"
Oct 30 18:19:01.728549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.728531    5746 scope.go:122] "RemoveContainer" containerID="3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d"
Oct 30 18:19:01.728828 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:19:01.728808    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d\": container with ID starting with 3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d not found: ID does not exist" containerID="3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d"
Oct 30 18:19:01.728892 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.728841    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d"} err="failed to get container status \"3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d\": rpc error: code = NotFound desc = could not find container \"3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d\": container with ID starting with 3bdf1d0aa98d64ee0d8bd4ae9bd9eb9b69b4d9c1156fd43f8f09fc757fa0b71d not found: ID does not exist"
Oct 30 18:19:01.768184 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.768160    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"]
Oct 30 18:19:01.779550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:01.779526    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5d979c9fdc-6qw49"]
Oct 30 18:19:02.007763 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:02.007678    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc946fb6-04a8-4f7e-81ec-724b2a63565c" path="/var/lib/kubelet/pods/fc946fb6-04a8-4f7e-81ec-724b2a63565c/volumes"
Oct 30 18:19:12.789364 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:19:12.789326    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:19:15.803564 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.803522    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"]
Oct 30 18:19:15.804109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.803838    5746 cpu_manager.go:414] "containerMap: removing container" podUID="fc946fb6-04a8-4f7e-81ec-724b2a63565c" containerName="ta-container"
Oct 30 18:19:15.804109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.803851    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc946fb6-04a8-4f7e-81ec-724b2a63565c" containerName="ta-container"
Oct 30 18:19:15.804109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.803943    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc946fb6-04a8-4f7e-81ec-724b2a63565c" containerName="ta-container"
Oct 30 18:19:15.808061 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.808040    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:15.835173 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.835147    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"]
Oct 30 18:19:15.885373 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.885339    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/d50bdcec-f983-4a24-b974-14e07db12243-ta-internal\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:15.885572 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.885375    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d50bdcec-f983-4a24-b974-14e07db12243-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:15.885572 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.885492    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjqb5\" (UniqueName: \"kubernetes.io/projected/d50bdcec-f983-4a24-b974-14e07db12243-kube-api-access-bjqb5\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:15.986128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.986093    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/d50bdcec-f983-4a24-b974-14e07db12243-ta-internal\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:15.986128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.986131    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d50bdcec-f983-4a24-b974-14e07db12243-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:15.986364 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.986161    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjqb5\" (UniqueName: \"kubernetes.io/projected/d50bdcec-f983-4a24-b974-14e07db12243-kube-api-access-bjqb5\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:15.986738 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.986717    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/d50bdcec-f983-4a24-b974-14e07db12243-ta-internal\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:15.988698 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:15.988675    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d50bdcec-f983-4a24-b974-14e07db12243-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:16.009347 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.009318    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjqb5\" (UniqueName: \"kubernetes.io/projected/d50bdcec-f983-4a24-b974-14e07db12243-kube-api-access-bjqb5\") pod \"data-science-collector-targetallocator-b6fd7d977-sp4wb\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:16.119058 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.119033    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:16.413751 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.413668    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"]
Oct 30 18:19:16.417170 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:19:16.417141    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd50bdcec_f983_4a24_b974_14e07db12243.slice/crio-93d81b78d2fe2f8ed4674fbf064f77fbbfa1bef407482094f5a631cc9a3b1217 WatchSource:0}: Error finding container 93d81b78d2fe2f8ed4674fbf064f77fbbfa1bef407482094f5a631cc9a3b1217: Status 404 returned error can't find the container with id 93d81b78d2fe2f8ed4674fbf064f77fbbfa1bef407482094f5a631cc9a3b1217
Oct 30 18:19:16.419525 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.419486    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:19:16.419598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.419562    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:19:16.781754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.781671    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb" event={"ID":"d50bdcec-f983-4a24-b974-14e07db12243","Type":"ContainerStarted","Data":"34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e"}
Oct 30 18:19:16.781754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.781713    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb" event={"ID":"d50bdcec-f983-4a24-b974-14e07db12243","Type":"ContainerStarted","Data":"93d81b78d2fe2f8ed4674fbf064f77fbbfa1bef407482094f5a631cc9a3b1217"}
Oct 30 18:19:16.781956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.781821    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:16.782920 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.782887    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb" podUID="d50bdcec-f983-4a24-b974-14e07db12243" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.40:8080/readyz\": dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:19:16.821940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:16.821852    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb" podStartSLOduration=1.821832957 podStartE2EDuration="1.821832957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:19:16.821467036 +0000 UTC m=+1326.988285163" watchObservedRunningTime="2026-10-30 18:19:16.821832957 +0000 UTC m=+1326.988651084"
Oct 30 18:19:17.787068 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:17.787041    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:17.859368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:17.859313    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"]
Oct 30 18:19:17.859854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:17.859536    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8" podUID="0074edeb-4e9b-45c6-89bc-ba91c3787ea1" containerName="ta-container" containerID="cri-o://5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74" gracePeriod=30
Oct 30 18:19:18.098161 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.098138    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:19:18.204630 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.204596    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-ta-internal\" (UniqueName: \"kubernetes.io/configmap/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-ta-internal\") pod \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") "
Oct 30 18:19:18.204800 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.204699    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-kube-api-access-8lm7r\" (UniqueName: \"kubernetes.io/projected/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-kube-api-access-8lm7r\") pod \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") "
Oct 30 18:19:18.204800 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.204726    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-data-science-collector-ta-server-cert\") pod \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\" (UID: \"0074edeb-4e9b-45c6-89bc-ba91c3787ea1\") "
Oct 30 18:19:18.204938 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.204894    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-ta-internal" pod "0074edeb-4e9b-45c6-89bc-ba91c3787ea1" (UID: "0074edeb-4e9b-45c6-89bc-ba91c3787ea1"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:19:18.206840 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.206818    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-kube-api-access-8lm7r" pod "0074edeb-4e9b-45c6-89bc-ba91c3787ea1" (UID: "0074edeb-4e9b-45c6-89bc-ba91c3787ea1"). InnerVolumeSpecName "kube-api-access-8lm7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:19:18.206978 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.206949    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-data-science-collector-ta-server-cert" pod "0074edeb-4e9b-45c6-89bc-ba91c3787ea1" (UID: "0074edeb-4e9b-45c6-89bc-ba91c3787ea1"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:19:18.305551 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.305510    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:18.305551 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.305543    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lm7r\" (UniqueName: \"kubernetes.io/projected/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-kube-api-access-8lm7r\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:18.305551 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.305554    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0074edeb-4e9b-45c6-89bc-ba91c3787ea1-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:18.790975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.790938    5746 generic.go:356] "Generic (PLEG): container finished" podID="0074edeb-4e9b-45c6-89bc-ba91c3787ea1" containerID="5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74" exitCode=0
Oct 30 18:19:18.791148 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.790982    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"
Oct 30 18:19:18.791148 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.791000    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8" event={"ID":"0074edeb-4e9b-45c6-89bc-ba91c3787ea1","Type":"ContainerDied","Data":"5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74"}
Oct 30 18:19:18.791148 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.791046    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8" event={"ID":"0074edeb-4e9b-45c6-89bc-ba91c3787ea1","Type":"ContainerDied","Data":"a8a1edf60858b376e4949491f3ceaa83dc825d07dd7955558dee7e06cd97ec8f"}
Oct 30 18:19:18.791148 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.791064    5746 scope.go:122] "RemoveContainer" containerID="5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74"
Oct 30 18:19:18.803856 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.803834    5746 scope.go:122] "RemoveContainer" containerID="5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74"
Oct 30 18:19:18.804259 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:19:18.804230    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74\": container with ID starting with 5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74 not found: ID does not exist" containerID="5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74"
Oct 30 18:19:18.804355 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.804270    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74"} err="failed to get container status \"5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74\": rpc error: code = NotFound desc = could not find container \"5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74\": container with ID starting with 5b2608c92aba6fe648eb1a73bdd9481710d06921d5a2a7c7148852ffb8b82f74 not found: ID does not exist"
Oct 30 18:19:18.839252 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.839226    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"]
Oct 30 18:19:18.850787 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:18.850754    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7f6dcd8b56-9vxr8"]
Oct 30 18:19:20.008671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:20.008640    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0074edeb-4e9b-45c6-89bc-ba91c3787ea1" path="/var/lib/kubelet/pods/0074edeb-4e9b-45c6-89bc-ba91c3787ea1/volumes"
Oct 30 18:19:33.581647 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.581611    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"]
Oct 30 18:19:33.582107 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.581923    5746 cpu_manager.go:414] "containerMap: removing container" podUID="0074edeb-4e9b-45c6-89bc-ba91c3787ea1" containerName="ta-container"
Oct 30 18:19:33.582107 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.581936    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0074edeb-4e9b-45c6-89bc-ba91c3787ea1" containerName="ta-container"
Oct 30 18:19:33.582107 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.581982    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="0074edeb-4e9b-45c6-89bc-ba91c3787ea1" containerName="ta-container"
Oct 30 18:19:33.584551 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.584535    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.611646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.611619    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"]
Oct 30 18:19:33.729194 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.729161    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm729\" (UniqueName: \"kubernetes.io/projected/fd494828-a644-48b6-b38d-fd82cc6a0cd0-kube-api-access-bm729\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.729194 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.729205    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fd494828-a644-48b6-b38d-fd82cc6a0cd0-ta-internal\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.729410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.729236    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fd494828-a644-48b6-b38d-fd82cc6a0cd0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.829640 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.829609    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fd494828-a644-48b6-b38d-fd82cc6a0cd0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.829798 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.829663    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm729\" (UniqueName: \"kubernetes.io/projected/fd494828-a644-48b6-b38d-fd82cc6a0cd0-kube-api-access-bm729\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.829798 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.829687    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fd494828-a644-48b6-b38d-fd82cc6a0cd0-ta-internal\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.830226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.830210    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fd494828-a644-48b6-b38d-fd82cc6a0cd0-ta-internal\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.832298 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.832256    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fd494828-a644-48b6-b38d-fd82cc6a0cd0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.852676 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.852650    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm729\" (UniqueName: \"kubernetes.io/projected/fd494828-a644-48b6-b38d-fd82cc6a0cd0-kube-api-access-bm729\") pod \"data-science-collector-targetallocator-d47567866-j4chm\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:33.895943 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:33.895901    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:34.190613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.190586    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"]
Oct 30 18:19:34.193297 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:19:34.193259    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd494828_a644_48b6_b38d_fd82cc6a0cd0.slice/crio-2898a1149a483ebc0e68002840228e149a0e571f8e5e4a84e6bac0ed6400672a WatchSource:0}: Error finding container 2898a1149a483ebc0e68002840228e149a0e571f8e5e4a84e6bac0ed6400672a: Status 404 returned error can't find the container with id 2898a1149a483ebc0e68002840228e149a0e571f8e5e4a84e6bac0ed6400672a
Oct 30 18:19:34.195560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.195524    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:19:34.195651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.195582    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:19:34.852355 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.852316    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm" event={"ID":"fd494828-a644-48b6-b38d-fd82cc6a0cd0","Type":"ContainerStarted","Data":"a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e"}
Oct 30 18:19:34.852799 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.852364    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm" event={"ID":"fd494828-a644-48b6-b38d-fd82cc6a0cd0","Type":"ContainerStarted","Data":"2898a1149a483ebc0e68002840228e149a0e571f8e5e4a84e6bac0ed6400672a"}
Oct 30 18:19:34.852799 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.852472    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:34.854398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.854377    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:34.935282 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.935228    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm" podStartSLOduration=1.935214244 podStartE2EDuration="1.935214244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:19:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:19:34.89462472 +0000 UTC m=+1345.061442849" watchObservedRunningTime="2026-10-30 18:19:34.935214244 +0000 UTC m=+1345.102032372"
Oct 30 18:19:34.964164 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.964136    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"]
Oct 30 18:19:34.964339 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:34.964313    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb" podUID="d50bdcec-f983-4a24-b974-14e07db12243" containerName="ta-container" containerID="cri-o://34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e" gracePeriod=30
Oct 30 18:19:35.199919 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.199882    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:35.341499 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.341468    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d50bdcec-f983-4a24-b974-14e07db12243-ta-internal\" (UniqueName: \"kubernetes.io/configmap/d50bdcec-f983-4a24-b974-14e07db12243-ta-internal\") pod \"d50bdcec-f983-4a24-b974-14e07db12243\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") "
Oct 30 18:19:35.341689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.341512    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d50bdcec-f983-4a24-b974-14e07db12243-kube-api-access-bjqb5\" (UniqueName: \"kubernetes.io/projected/d50bdcec-f983-4a24-b974-14e07db12243-kube-api-access-bjqb5\") pod \"d50bdcec-f983-4a24-b974-14e07db12243\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") "
Oct 30 18:19:35.341689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.341564    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d50bdcec-f983-4a24-b974-14e07db12243-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d50bdcec-f983-4a24-b974-14e07db12243-data-science-collector-ta-server-cert\") pod \"d50bdcec-f983-4a24-b974-14e07db12243\" (UID: \"d50bdcec-f983-4a24-b974-14e07db12243\") "
Oct 30 18:19:35.341868 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.341839    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d50bdcec-f983-4a24-b974-14e07db12243-ta-internal" pod "d50bdcec-f983-4a24-b974-14e07db12243" (UID: "d50bdcec-f983-4a24-b974-14e07db12243"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:19:35.343797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.343767    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d50bdcec-f983-4a24-b974-14e07db12243-data-science-collector-ta-server-cert" pod "d50bdcec-f983-4a24-b974-14e07db12243" (UID: "d50bdcec-f983-4a24-b974-14e07db12243"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:19:35.343897 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.343807    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d50bdcec-f983-4a24-b974-14e07db12243-kube-api-access-bjqb5" pod "d50bdcec-f983-4a24-b974-14e07db12243" (UID: "d50bdcec-f983-4a24-b974-14e07db12243"). InnerVolumeSpecName "kube-api-access-bjqb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:19:35.442760 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.442681    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/d50bdcec-f983-4a24-b974-14e07db12243-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:35.442760 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.442711    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjqb5\" (UniqueName: \"kubernetes.io/projected/d50bdcec-f983-4a24-b974-14e07db12243-kube-api-access-bjqb5\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:35.442760 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.442721    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d50bdcec-f983-4a24-b974-14e07db12243-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:35.856992 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.856958    5746 generic.go:356] "Generic (PLEG): container finished" podID="d50bdcec-f983-4a24-b974-14e07db12243" containerID="34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e" exitCode=0
Oct 30 18:19:35.857463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.857005    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"
Oct 30 18:19:35.857463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.857047    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb" event={"ID":"d50bdcec-f983-4a24-b974-14e07db12243","Type":"ContainerDied","Data":"34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e"}
Oct 30 18:19:35.857463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.857094    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb" event={"ID":"d50bdcec-f983-4a24-b974-14e07db12243","Type":"ContainerDied","Data":"93d81b78d2fe2f8ed4674fbf064f77fbbfa1bef407482094f5a631cc9a3b1217"}
Oct 30 18:19:35.857463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.857111    5746 scope.go:122] "RemoveContainer" containerID="34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e"
Oct 30 18:19:35.866185 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.866166    5746 scope.go:122] "RemoveContainer" containerID="34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e"
Oct 30 18:19:35.866458 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:19:35.866437    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e\": container with ID starting with 34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e not found: ID does not exist" containerID="34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e"
Oct 30 18:19:35.866508 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.866468    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e"} err="failed to get container status \"34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e\": rpc error: code = NotFound desc = could not find container \"34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e\": container with ID starting with 34fb7fb3de89bea3118f866ae772466da60cde100aa4f105ffa612b97c22055e not found: ID does not exist"
Oct 30 18:19:35.906756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.906719    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"]
Oct 30 18:19:35.918276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:35.918245    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-b6fd7d977-sp4wb"]
Oct 30 18:19:36.008570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:36.008532    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d50bdcec-f983-4a24-b974-14e07db12243" path="/var/lib/kubelet/pods/d50bdcec-f983-4a24-b974-14e07db12243/volumes"
Oct 30 18:19:42.789855 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:19:42.789814    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:19:51.034872 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.034826    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"]
Oct 30 18:19:51.035487 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.035265    5746 cpu_manager.go:414] "containerMap: removing container" podUID="d50bdcec-f983-4a24-b974-14e07db12243" containerName="ta-container"
Oct 30 18:19:51.035487 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.035282    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d50bdcec-f983-4a24-b974-14e07db12243" containerName="ta-container"
Oct 30 18:19:51.035487 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.035369    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="d50bdcec-f983-4a24-b974-14e07db12243" containerName="ta-container"
Oct 30 18:19:51.039588 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.039566    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.066613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.066588    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"]
Oct 30 18:19:51.172656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.172626    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn9cd\" (UniqueName: \"kubernetes.io/projected/0ce088c3-44f3-40c0-9bc5-3e3133906209-kube-api-access-xn9cd\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.172808 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.172678    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/0ce088c3-44f3-40c0-9bc5-3e3133906209-ta-internal\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.172808 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.172750    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0ce088c3-44f3-40c0-9bc5-3e3133906209-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.273383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.273346    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0ce088c3-44f3-40c0-9bc5-3e3133906209-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.273560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.273409    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn9cd\" (UniqueName: \"kubernetes.io/projected/0ce088c3-44f3-40c0-9bc5-3e3133906209-kube-api-access-xn9cd\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.273560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.273447    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/0ce088c3-44f3-40c0-9bc5-3e3133906209-ta-internal\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.274018 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.273999    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/0ce088c3-44f3-40c0-9bc5-3e3133906209-ta-internal\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.276053 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.276029    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0ce088c3-44f3-40c0-9bc5-3e3133906209-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.296391 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.296339    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn9cd\" (UniqueName: \"kubernetes.io/projected/0ce088c3-44f3-40c0-9bc5-3e3133906209-kube-api-access-xn9cd\") pod \"data-science-collector-targetallocator-8585659779-5bcb9\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.350602 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.350578    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.646276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.646242    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"]
Oct 30 18:19:51.649276 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:19:51.649246    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ce088c3_44f3_40c0_9bc5_3e3133906209.slice/crio-c591d36c87b6f02c737f1194059dffac18e812a63da5199715a0bbf0a3d60d39 WatchSource:0}: Error finding container c591d36c87b6f02c737f1194059dffac18e812a63da5199715a0bbf0a3d60d39: Status 404 returned error can't find the container with id c591d36c87b6f02c737f1194059dffac18e812a63da5199715a0bbf0a3d60d39
Oct 30 18:19:51.651701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.651671    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:19:51.651769 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.651729    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:19:51.921997 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.921900    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9" event={"ID":"0ce088c3-44f3-40c0-9bc5-3e3133906209","Type":"ContainerStarted","Data":"e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699"}
Oct 30 18:19:51.921997 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.921962    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9" event={"ID":"0ce088c3-44f3-40c0-9bc5-3e3133906209","Type":"ContainerStarted","Data":"c591d36c87b6f02c737f1194059dffac18e812a63da5199715a0bbf0a3d60d39"}
Oct 30 18:19:51.922183 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.922090    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:51.923088 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.923061    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9" podUID="0ce088c3-44f3-40c0-9bc5-3e3133906209" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.42:8080/readyz\": dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 18:19:51.962037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:51.961996    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9" podStartSLOduration=0.961984041 podStartE2EDuration="961.984041ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:19:51.960978622 +0000 UTC m=+1362.127796749" watchObservedRunningTime="2026-10-30 18:19:51.961984041 +0000 UTC m=+1362.128802168"
Oct 30 18:19:52.926237 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:52.926205    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:19:52.995705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:52.995676    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"]
Oct 30 18:19:52.995870 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:52.995850    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm" podUID="fd494828-a644-48b6-b38d-fd82cc6a0cd0" containerName="ta-container" containerID="cri-o://a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e" gracePeriod=30
Oct 30 18:19:53.233114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.233090    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:53.392620 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.392586    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd494828-a644-48b6-b38d-fd82cc6a0cd0-ta-internal\" (UniqueName: \"kubernetes.io/configmap/fd494828-a644-48b6-b38d-fd82cc6a0cd0-ta-internal\") pod \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") "
Oct 30 18:19:53.392783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.392677    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd494828-a644-48b6-b38d-fd82cc6a0cd0-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fd494828-a644-48b6-b38d-fd82cc6a0cd0-data-science-collector-ta-server-cert\") pod \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") "
Oct 30 18:19:53.392783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.392770    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd494828-a644-48b6-b38d-fd82cc6a0cd0-kube-api-access-bm729\" (UniqueName: \"kubernetes.io/projected/fd494828-a644-48b6-b38d-fd82cc6a0cd0-kube-api-access-bm729\") pod \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\" (UID: \"fd494828-a644-48b6-b38d-fd82cc6a0cd0\") "
Oct 30 18:19:53.393041 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.393021    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd494828-a644-48b6-b38d-fd82cc6a0cd0-ta-internal" pod "fd494828-a644-48b6-b38d-fd82cc6a0cd0" (UID: "fd494828-a644-48b6-b38d-fd82cc6a0cd0"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:19:53.395028 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.395002    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd494828-a644-48b6-b38d-fd82cc6a0cd0-data-science-collector-ta-server-cert" pod "fd494828-a644-48b6-b38d-fd82cc6a0cd0" (UID: "fd494828-a644-48b6-b38d-fd82cc6a0cd0"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:19:53.395028 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.395007    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd494828-a644-48b6-b38d-fd82cc6a0cd0-kube-api-access-bm729" pod "fd494828-a644-48b6-b38d-fd82cc6a0cd0" (UID: "fd494828-a644-48b6-b38d-fd82cc6a0cd0"). InnerVolumeSpecName "kube-api-access-bm729". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:19:53.493837 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.493763    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fd494828-a644-48b6-b38d-fd82cc6a0cd0-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:53.493837 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.493793    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm729\" (UniqueName: \"kubernetes.io/projected/fd494828-a644-48b6-b38d-fd82cc6a0cd0-kube-api-access-bm729\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:53.493837 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.493802    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fd494828-a644-48b6-b38d-fd82cc6a0cd0-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:19:53.930341 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.930304    5746 generic.go:356] "Generic (PLEG): container finished" podID="fd494828-a644-48b6-b38d-fd82cc6a0cd0" containerID="a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e" exitCode=0
Oct 30 18:19:53.930712 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.930360    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"
Oct 30 18:19:53.930712 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.930387    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm" event={"ID":"fd494828-a644-48b6-b38d-fd82cc6a0cd0","Type":"ContainerDied","Data":"a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e"}
Oct 30 18:19:53.930712 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.930426    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm" event={"ID":"fd494828-a644-48b6-b38d-fd82cc6a0cd0","Type":"ContainerDied","Data":"2898a1149a483ebc0e68002840228e149a0e571f8e5e4a84e6bac0ed6400672a"}
Oct 30 18:19:53.930712 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.930456    5746 scope.go:122] "RemoveContainer" containerID="a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e"
Oct 30 18:19:53.938966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.938896    5746 scope.go:122] "RemoveContainer" containerID="a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e"
Oct 30 18:19:53.939252 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:19:53.939232    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e\": container with ID starting with a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e not found: ID does not exist" containerID="a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e"
Oct 30 18:19:53.939316 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.939265    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e"} err="failed to get container status \"a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e\": rpc error: code = NotFound desc = could not find container \"a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e\": container with ID starting with a895187cf1100e1dc888f0c95066ecf39e6ab4367db093d7683cefe4f23cab2e not found: ID does not exist"
Oct 30 18:19:53.982258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.982228    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"]
Oct 30 18:19:53.993637 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:53.993616    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-d47567866-j4chm"]
Oct 30 18:19:54.007968 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:19:54.007943    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd494828-a644-48b6-b38d-fd82cc6a0cd0" path="/var/lib/kubelet/pods/fd494828-a644-48b6-b38d-fd82cc6a0cd0/volumes"
Oct 30 18:20:08.375392 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.375356    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"]
Oct 30 18:20:08.375821 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.375767    5746 cpu_manager.go:414] "containerMap: removing container" podUID="fd494828-a644-48b6-b38d-fd82cc6a0cd0" containerName="ta-container"
Oct 30 18:20:08.375821 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.375782    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd494828-a644-48b6-b38d-fd82cc6a0cd0" containerName="ta-container"
Oct 30 18:20:08.375894 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.375837    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd494828-a644-48b6-b38d-fd82cc6a0cd0" containerName="ta-container"
Oct 30 18:20:08.378591 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.378577    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.394545 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.394525    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9fbg\" (UniqueName: \"kubernetes.io/projected/b868d40a-544a-4ebd-957e-ce884480f999-kube-api-access-d9fbg\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.394671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.394564    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/b868d40a-544a-4ebd-957e-ce884480f999-ta-internal\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.394671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.394599    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b868d40a-544a-4ebd-957e-ce884480f999-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.409781 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.409758    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"]
Oct 30 18:20:08.496051 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.496016    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9fbg\" (UniqueName: \"kubernetes.io/projected/b868d40a-544a-4ebd-957e-ce884480f999-kube-api-access-d9fbg\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.496203 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.496070    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/b868d40a-544a-4ebd-957e-ce884480f999-ta-internal\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.496203 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.496097    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b868d40a-544a-4ebd-957e-ce884480f999-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.496897 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.496875    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/b868d40a-544a-4ebd-957e-ce884480f999-ta-internal\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.498613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.498594    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b868d40a-544a-4ebd-957e-ce884480f999-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.518782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.518759    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9fbg\" (UniqueName: \"kubernetes.io/projected/b868d40a-544a-4ebd-957e-ce884480f999-kube-api-access-d9fbg\") pod \"data-science-collector-targetallocator-77d94746b8-m66zk\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.689951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.689847    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:08.983861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.983789    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"]
Oct 30 18:20:08.987219 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:20:08.987187    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb868d40a_544a_4ebd_957e_ce884480f999.slice/crio-843dac08a4cb40da6dcb6e9614e475a8a82e4250359848a7f7de81ac463721af WatchSource:0}: Error finding container 843dac08a4cb40da6dcb6e9614e475a8a82e4250359848a7f7de81ac463721af: Status 404 returned error can't find the container with id 843dac08a4cb40da6dcb6e9614e475a8a82e4250359848a7f7de81ac463721af
Oct 30 18:20:08.989605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.989570    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:20:08.989703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.989640    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:20:08.997960 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:08.997937    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk" event={"ID":"b868d40a-544a-4ebd-957e-ce884480f999","Type":"ContainerStarted","Data":"843dac08a4cb40da6dcb6e9614e475a8a82e4250359848a7f7de81ac463721af"}
Oct 30 18:20:10.007750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.007717    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk" event={"ID":"b868d40a-544a-4ebd-957e-ce884480f999","Type":"ContainerStarted","Data":"dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0"}
Oct 30 18:20:10.008170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.007762    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:10.008170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.007793    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:10.045606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.045407    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk" podStartSLOduration=2.045389768 podStartE2EDuration="2.045389768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:20:10.044248134 +0000 UTC m=+1380.211066274" watchObservedRunningTime="2026-10-30 18:20:10.045389768 +0000 UTC m=+1380.212207898"
Oct 30 18:20:10.115343 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.115317    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"]
Oct 30 18:20:10.115560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.115538    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9" podUID="0ce088c3-44f3-40c0-9bc5-3e3133906209" containerName="ta-container" containerID="cri-o://e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699" gracePeriod=30
Oct 30 18:20:10.359876 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.359855    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:20:10.408991 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.408958    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ce088c3-44f3-40c0-9bc5-3e3133906209-ta-internal\" (UniqueName: \"kubernetes.io/configmap/0ce088c3-44f3-40c0-9bc5-3e3133906209-ta-internal\") pod \"0ce088c3-44f3-40c0-9bc5-3e3133906209\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") "
Oct 30 18:20:10.409157 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.409028    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ce088c3-44f3-40c0-9bc5-3e3133906209-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0ce088c3-44f3-40c0-9bc5-3e3133906209-data-science-collector-ta-server-cert\") pod \"0ce088c3-44f3-40c0-9bc5-3e3133906209\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") "
Oct 30 18:20:10.409157 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.409071    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ce088c3-44f3-40c0-9bc5-3e3133906209-kube-api-access-xn9cd\" (UniqueName: \"kubernetes.io/projected/0ce088c3-44f3-40c0-9bc5-3e3133906209-kube-api-access-xn9cd\") pod \"0ce088c3-44f3-40c0-9bc5-3e3133906209\" (UID: \"0ce088c3-44f3-40c0-9bc5-3e3133906209\") "
Oct 30 18:20:10.409298 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.409274    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ce088c3-44f3-40c0-9bc5-3e3133906209-ta-internal" pod "0ce088c3-44f3-40c0-9bc5-3e3133906209" (UID: "0ce088c3-44f3-40c0-9bc5-3e3133906209"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:20:10.411290 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.411266    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ce088c3-44f3-40c0-9bc5-3e3133906209-data-science-collector-ta-server-cert" pod "0ce088c3-44f3-40c0-9bc5-3e3133906209" (UID: "0ce088c3-44f3-40c0-9bc5-3e3133906209"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:20:10.411406 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.411388    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ce088c3-44f3-40c0-9bc5-3e3133906209-kube-api-access-xn9cd" pod "0ce088c3-44f3-40c0-9bc5-3e3133906209" (UID: "0ce088c3-44f3-40c0-9bc5-3e3133906209"). InnerVolumeSpecName "kube-api-access-xn9cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:20:10.509815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.509782    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn9cd\" (UniqueName: \"kubernetes.io/projected/0ce088c3-44f3-40c0-9bc5-3e3133906209-kube-api-access-xn9cd\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:10.509815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.509806    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/0ce088c3-44f3-40c0-9bc5-3e3133906209-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:10.509815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:10.509817    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/0ce088c3-44f3-40c0-9bc5-3e3133906209-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:11.008573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.008535    5746 generic.go:356] "Generic (PLEG): container finished" podID="0ce088c3-44f3-40c0-9bc5-3e3133906209" containerID="e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699" exitCode=0
Oct 30 18:20:11.009016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.008580    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"
Oct 30 18:20:11.009016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.008626    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9" event={"ID":"0ce088c3-44f3-40c0-9bc5-3e3133906209","Type":"ContainerDied","Data":"e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699"}
Oct 30 18:20:11.009016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.008669    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9" event={"ID":"0ce088c3-44f3-40c0-9bc5-3e3133906209","Type":"ContainerDied","Data":"c591d36c87b6f02c737f1194059dffac18e812a63da5199715a0bbf0a3d60d39"}
Oct 30 18:20:11.009016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.008693    5746 scope.go:122] "RemoveContainer" containerID="e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699"
Oct 30 18:20:11.017494 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.017472    5746 scope.go:122] "RemoveContainer" containerID="e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699"
Oct 30 18:20:11.017782 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:20:11.017764    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699\": container with ID starting with e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699 not found: ID does not exist" containerID="e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699"
Oct 30 18:20:11.017849 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.017796    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699"} err="failed to get container status \"e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699\": rpc error: code = NotFound desc = could not find container \"e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699\": container with ID starting with e2cdfadc01e384ce95b083f030afa849cbac75fd65e8cc8ee657814568a1f699 not found: ID does not exist"
Oct 30 18:20:11.057035 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.057013    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"]
Oct 30 18:20:11.068629 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:11.068607    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-8585659779-5bcb9"]
Oct 30 18:20:12.007924 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:12.007873    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ce088c3-44f3-40c0-9bc5-3e3133906209" path="/var/lib/kubelet/pods/0ce088c3-44f3-40c0-9bc5-3e3133906209/volumes"
Oct 30 18:20:12.790288 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:20:12.790246    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:20:25.040930 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.040878    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"]
Oct 30 18:20:25.041437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.041221    5746 cpu_manager.go:414] "containerMap: removing container" podUID="0ce088c3-44f3-40c0-9bc5-3e3133906209" containerName="ta-container"
Oct 30 18:20:25.041437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.041233    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ce088c3-44f3-40c0-9bc5-3e3133906209" containerName="ta-container"
Oct 30 18:20:25.041437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.041285    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ce088c3-44f3-40c0-9bc5-3e3133906209" containerName="ta-container"
Oct 30 18:20:25.045552 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.045535    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.072378 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.072353    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"]
Oct 30 18:20:25.121636 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.121607    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnmjc\" (UniqueName: \"kubernetes.io/projected/cd86ddfd-979a-408f-827e-1c3b3463f84b-kube-api-access-xnmjc\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.121753 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.121656    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cd86ddfd-979a-408f-827e-1c3b3463f84b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.121753 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.121676    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/cd86ddfd-979a-408f-827e-1c3b3463f84b-ta-internal\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.222596 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.222555    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/cd86ddfd-979a-408f-827e-1c3b3463f84b-ta-internal\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.222747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.222620    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnmjc\" (UniqueName: \"kubernetes.io/projected/cd86ddfd-979a-408f-827e-1c3b3463f84b-kube-api-access-xnmjc\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.222747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.222667    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cd86ddfd-979a-408f-827e-1c3b3463f84b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.223309 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.223287    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/cd86ddfd-979a-408f-827e-1c3b3463f84b-ta-internal\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.225122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.225106    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cd86ddfd-979a-408f-827e-1c3b3463f84b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.245684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.245654    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnmjc\" (UniqueName: \"kubernetes.io/projected/cd86ddfd-979a-408f-827e-1c3b3463f84b-kube-api-access-xnmjc\") pod \"data-science-collector-targetallocator-57f9d9dd69-8wvgz\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.357328 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.357305    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:25.653131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.653061    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"]
Oct 30 18:20:25.656045 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:20:25.656006    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd86ddfd_979a_408f_827e_1c3b3463f84b.slice/crio-56f923a14d5fec6a582f134fdb49eade64067fe81dd099b1b33cf0e048af30d5 WatchSource:0}: Error finding container 56f923a14d5fec6a582f134fdb49eade64067fe81dd099b1b33cf0e048af30d5: Status 404 returned error can't find the container with id 56f923a14d5fec6a582f134fdb49eade64067fe81dd099b1b33cf0e048af30d5
Oct 30 18:20:25.658276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.658243    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:20:25.658384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:25.658307    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:20:26.070480 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:26.070385    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz" event={"ID":"cd86ddfd-979a-408f-827e-1c3b3463f84b","Type":"ContainerStarted","Data":"50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0"}
Oct 30 18:20:26.070480 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:26.070428    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz" event={"ID":"cd86ddfd-979a-408f-827e-1c3b3463f84b","Type":"ContainerStarted","Data":"56f923a14d5fec6a582f134fdb49eade64067fe81dd099b1b33cf0e048af30d5"}
Oct 30 18:20:26.071009 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:26.070508    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:26.071672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:26.071646    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz" podUID="cd86ddfd-979a-408f-827e-1c3b3463f84b" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.44:8080/readyz\": dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 18:20:26.111784 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:26.111732    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz" podStartSLOduration=1.11171794 podStartE2EDuration="1.11171794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:20:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:20:26.11070204 +0000 UTC m=+1396.277520188" watchObservedRunningTime="2026-10-30 18:20:26.11171794 +0000 UTC m=+1396.278536124"
Oct 30 18:20:27.075761 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.075728    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:27.145395 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.145358    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"]
Oct 30 18:20:27.145605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.145580    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk" podUID="b868d40a-544a-4ebd-957e-ce884480f999" containerName="ta-container" containerID="cri-o://dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0" gracePeriod=30
Oct 30 18:20:27.382193 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.382167    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:27.440792 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.440765    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b868d40a-544a-4ebd-957e-ce884480f999-kube-api-access-d9fbg\" (UniqueName: \"kubernetes.io/projected/b868d40a-544a-4ebd-957e-ce884480f999-kube-api-access-d9fbg\") pod \"b868d40a-544a-4ebd-957e-ce884480f999\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") "
Oct 30 18:20:27.440992 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.440807    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b868d40a-544a-4ebd-957e-ce884480f999-ta-internal\" (UniqueName: \"kubernetes.io/configmap/b868d40a-544a-4ebd-957e-ce884480f999-ta-internal\") pod \"b868d40a-544a-4ebd-957e-ce884480f999\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") "
Oct 30 18:20:27.440992 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.440867    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b868d40a-544a-4ebd-957e-ce884480f999-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b868d40a-544a-4ebd-957e-ce884480f999-data-science-collector-ta-server-cert\") pod \"b868d40a-544a-4ebd-957e-ce884480f999\" (UID: \"b868d40a-544a-4ebd-957e-ce884480f999\") "
Oct 30 18:20:27.441241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.441206    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b868d40a-544a-4ebd-957e-ce884480f999-ta-internal" pod "b868d40a-544a-4ebd-957e-ce884480f999" (UID: "b868d40a-544a-4ebd-957e-ce884480f999"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:20:27.443165 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.443141    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b868d40a-544a-4ebd-957e-ce884480f999-data-science-collector-ta-server-cert" pod "b868d40a-544a-4ebd-957e-ce884480f999" (UID: "b868d40a-544a-4ebd-957e-ce884480f999"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:20:27.443251 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.443208    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b868d40a-544a-4ebd-957e-ce884480f999-kube-api-access-d9fbg" pod "b868d40a-544a-4ebd-957e-ce884480f999" (UID: "b868d40a-544a-4ebd-957e-ce884480f999"). InnerVolumeSpecName "kube-api-access-d9fbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:20:27.541883 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.541856    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b868d40a-544a-4ebd-957e-ce884480f999-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:27.541883 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.541880    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9fbg\" (UniqueName: \"kubernetes.io/projected/b868d40a-544a-4ebd-957e-ce884480f999-kube-api-access-d9fbg\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:27.541883 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:27.541889    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/b868d40a-544a-4ebd-957e-ce884480f999-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:28.079743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.079712    5746 generic.go:356] "Generic (PLEG): container finished" podID="b868d40a-544a-4ebd-957e-ce884480f999" containerID="dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0" exitCode=0
Oct 30 18:20:28.080234 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.079753    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"
Oct 30 18:20:28.080234 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.079802    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk" event={"ID":"b868d40a-544a-4ebd-957e-ce884480f999","Type":"ContainerDied","Data":"dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0"}
Oct 30 18:20:28.080234 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.079849    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk" event={"ID":"b868d40a-544a-4ebd-957e-ce884480f999","Type":"ContainerDied","Data":"843dac08a4cb40da6dcb6e9614e475a8a82e4250359848a7f7de81ac463721af"}
Oct 30 18:20:28.080234 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.079869    5746 scope.go:122] "RemoveContainer" containerID="dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0"
Oct 30 18:20:28.087752 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.087738    5746 scope.go:122] "RemoveContainer" containerID="dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0"
Oct 30 18:20:28.088020 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:20:28.088000    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0\": container with ID starting with dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0 not found: ID does not exist" containerID="dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0"
Oct 30 18:20:28.088090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.088032    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0"} err="failed to get container status \"dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0\": rpc error: code = NotFound desc = could not find container \"dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0\": container with ID starting with dd1b70986b8fd00d27e97ae54e4a2b110f6b3f3d7c29752e806d4279c62fd8b0 not found: ID does not exist"
Oct 30 18:20:28.130016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.129983    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"]
Oct 30 18:20:28.141780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:28.141750    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77d94746b8-m66zk"]
Oct 30 18:20:30.012078 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:30.009325    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b868d40a-544a-4ebd-957e-ce884480f999" path="/var/lib/kubelet/pods/b868d40a-544a-4ebd-957e-ce884480f999/volumes"
Oct 30 18:20:42.200047 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.200011    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-2"]
Oct 30 18:20:42.200497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.200345    5746 cpu_manager.go:414] "containerMap: removing container" podUID="b868d40a-544a-4ebd-957e-ce884480f999" containerName="ta-container"
Oct 30 18:20:42.200497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.200359    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b868d40a-544a-4ebd-957e-ce884480f999" containerName="ta-container"
Oct 30 18:20:42.200497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.200441    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="b868d40a-544a-4ebd-957e-ce884480f999" containerName="ta-container"
Oct 30 18:20:42.204731 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.204709    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.231742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.231581    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsdwv\""
Oct 30 18:20:42.231742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.231599    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-a0f9182a\""
Oct 30 18:20:42.231742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.231666    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:20:42.231742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.231671    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:20:42.231742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.231714    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:20:42.232073 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.231768    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:20:42.244228 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.244200    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-2"]
Oct 30 18:20:42.256285 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.256261    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.256418 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.256293    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-monitoring-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.256418 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.256312    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otlp-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.256418 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.256336    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g6hf\" (UniqueName: \"kubernetes.io/projected/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-kube-api-access-5g6hf\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.256562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.256522    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.256610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.256574    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otc-internal\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.357414 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.357385    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.357414 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.357415    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-monitoring-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.357673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.357433    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otlp-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.357673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.357460    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g6hf\" (UniqueName: \"kubernetes.io/projected/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-kube-api-access-5g6hf\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.357673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.357545    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.357673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.357581    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otc-internal\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.358548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.358519    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otc-internal\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.360190 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.360162    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-monitoring-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.360295 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.360242    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otlp-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.360295 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.360263    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-tls-certs\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.360371 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.360357    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.378041 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.378019    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"]
Oct 30 18:20:42.380886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.380861    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g6hf\" (UniqueName: \"kubernetes.io/projected/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-kube-api-access-5g6hf\") pod \"data-science-collector-collector-2\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") " pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.381155 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.381141    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.402443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.402416    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"]
Oct 30 18:20:42.458632 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.458558    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/e1d550bd-15e0-49f9-9c0b-556d480de831-ta-internal\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.458632 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.458594    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmj9z\" (UniqueName: \"kubernetes.io/projected/e1d550bd-15e0-49f9-9c0b-556d480de831-kube-api-access-mmj9z\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.458812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.458747    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e1d550bd-15e0-49f9-9c0b-556d480de831-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.517167 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.517134    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:20:42.559254 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.559218    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/e1d550bd-15e0-49f9-9c0b-556d480de831-ta-internal\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.559427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.559266    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmj9z\" (UniqueName: \"kubernetes.io/projected/e1d550bd-15e0-49f9-9c0b-556d480de831-kube-api-access-mmj9z\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.559427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.559358    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e1d550bd-15e0-49f9-9c0b-556d480de831-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.559948 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.559897    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/e1d550bd-15e0-49f9-9c0b-556d480de831-ta-internal\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.562060 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.562035    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e1d550bd-15e0-49f9-9c0b-556d480de831-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.582709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.582681    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmj9z\" (UniqueName: \"kubernetes.io/projected/e1d550bd-15e0-49f9-9c0b-556d480de831-kube-api-access-mmj9z\") pod \"data-science-collector-targetallocator-6db65bc7b6-7pp4m\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.696676 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.696640    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:42.792541 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:20:42.792504    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:20:42.815349 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.815315    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-2"]
Oct 30 18:20:42.818136 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:20:42.818109    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f05e33_3be6_43f7_9696_09de0b1a8aa0.slice/crio-062ad6adb02fe4a0ec8b63214c38b2d016345a653f9cd30575c157ac1cddd55b WatchSource:0}: Error finding container 062ad6adb02fe4a0ec8b63214c38b2d016345a653f9cd30575c157ac1cddd55b: Status 404 returned error can't find the container with id 062ad6adb02fe4a0ec8b63214c38b2d016345a653f9cd30575c157ac1cddd55b
Oct 30 18:20:42.820238 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.820220    5746 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:20:42.991755 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.991720    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"]
Oct 30 18:20:42.994590 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:20:42.994565    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1d550bd_15e0_49f9_9c0b_556d480de831.slice/crio-a19f0cb137d387b566f78b3aeea23d15a5337856127cf5d925ecfce05a91ba30 WatchSource:0}: Error finding container a19f0cb137d387b566f78b3aeea23d15a5337856127cf5d925ecfce05a91ba30: Status 404 returned error can't find the container with id a19f0cb137d387b566f78b3aeea23d15a5337856127cf5d925ecfce05a91ba30
Oct 30 18:20:42.996684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.996654    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:20:42.996766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:42.996708    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:20:43.138367 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:43.138331    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m" event={"ID":"e1d550bd-15e0-49f9-9c0b-556d480de831","Type":"ContainerStarted","Data":"c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df"}
Oct 30 18:20:43.138367 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:43.138373    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m" event={"ID":"e1d550bd-15e0-49f9-9c0b-556d480de831","Type":"ContainerStarted","Data":"a19f0cb137d387b566f78b3aeea23d15a5337856127cf5d925ecfce05a91ba30"}
Oct 30 18:20:43.138626 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:43.138497    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:43.139731 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:43.139698    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m" podUID="e1d550bd-15e0-49f9-9c0b-556d480de831" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.46:8080/readyz\": dial tcp 10.133.0.46:8080: connect: connection refused"
Oct 30 18:20:43.139861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:43.139787    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-2" event={"ID":"f7f05e33-3be6-43f7-9696-09de0b1a8aa0","Type":"ContainerStarted","Data":"062ad6adb02fe4a0ec8b63214c38b2d016345a653f9cd30575c157ac1cddd55b"}
Oct 30 18:20:43.179033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:43.178981    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m" podStartSLOduration=1.178963939 podStartE2EDuration="1.178963939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:20:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:20:43.178579355 +0000 UTC m=+1413.345397498" watchObservedRunningTime="2026-10-30 18:20:43.178963939 +0000 UTC m=+1413.345782068"
Oct 30 18:20:44.146206 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.146172    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:20:44.215923 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.215870    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"]
Oct 30 18:20:44.216115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.216094    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz" podUID="cd86ddfd-979a-408f-827e-1c3b3463f84b" containerName="ta-container" containerID="cri-o://50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0" gracePeriod=30
Oct 30 18:20:44.469628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.469603    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:44.575017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.574981    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd86ddfd-979a-408f-827e-1c3b3463f84b-ta-internal\" (UniqueName: \"kubernetes.io/configmap/cd86ddfd-979a-408f-827e-1c3b3463f84b-ta-internal\") pod \"cd86ddfd-979a-408f-827e-1c3b3463f84b\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") "
Oct 30 18:20:44.575269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.575084    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd86ddfd-979a-408f-827e-1c3b3463f84b-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cd86ddfd-979a-408f-827e-1c3b3463f84b-data-science-collector-ta-server-cert\") pod \"cd86ddfd-979a-408f-827e-1c3b3463f84b\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") "
Oct 30 18:20:44.575269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.575132    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd86ddfd-979a-408f-827e-1c3b3463f84b-kube-api-access-xnmjc\" (UniqueName: \"kubernetes.io/projected/cd86ddfd-979a-408f-827e-1c3b3463f84b-kube-api-access-xnmjc\") pod \"cd86ddfd-979a-408f-827e-1c3b3463f84b\" (UID: \"cd86ddfd-979a-408f-827e-1c3b3463f84b\") "
Oct 30 18:20:44.575452 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.575426    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd86ddfd-979a-408f-827e-1c3b3463f84b-ta-internal" pod "cd86ddfd-979a-408f-827e-1c3b3463f84b" (UID: "cd86ddfd-979a-408f-827e-1c3b3463f84b"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:20:44.577508 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.577482    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd86ddfd-979a-408f-827e-1c3b3463f84b-kube-api-access-xnmjc" pod "cd86ddfd-979a-408f-827e-1c3b3463f84b" (UID: "cd86ddfd-979a-408f-827e-1c3b3463f84b"). InnerVolumeSpecName "kube-api-access-xnmjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:20:44.577646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.577622    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd86ddfd-979a-408f-827e-1c3b3463f84b-data-science-collector-ta-server-cert" pod "cd86ddfd-979a-408f-827e-1c3b3463f84b" (UID: "cd86ddfd-979a-408f-827e-1c3b3463f84b"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:20:44.675803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.675766    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/cd86ddfd-979a-408f-827e-1c3b3463f84b-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:44.675803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.675796    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/cd86ddfd-979a-408f-827e-1c3b3463f84b-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:44.675803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:44.675807    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnmjc\" (UniqueName: \"kubernetes.io/projected/cd86ddfd-979a-408f-827e-1c3b3463f84b-kube-api-access-xnmjc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:20:45.150775 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.150737    5746 generic.go:356] "Generic (PLEG): container finished" podID="cd86ddfd-979a-408f-827e-1c3b3463f84b" containerID="50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0" exitCode=0
Oct 30 18:20:45.151286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.150783    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"
Oct 30 18:20:45.151286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.150825    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz" event={"ID":"cd86ddfd-979a-408f-827e-1c3b3463f84b","Type":"ContainerDied","Data":"50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0"}
Oct 30 18:20:45.151286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.150877    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz" event={"ID":"cd86ddfd-979a-408f-827e-1c3b3463f84b","Type":"ContainerDied","Data":"56f923a14d5fec6a582f134fdb49eade64067fe81dd099b1b33cf0e048af30d5"}
Oct 30 18:20:45.151286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.150901    5746 scope.go:122] "RemoveContainer" containerID="50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0"
Oct 30 18:20:45.160151 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.160127    5746 scope.go:122] "RemoveContainer" containerID="50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0"
Oct 30 18:20:45.160590 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:20:45.160494    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0\": container with ID starting with 50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0 not found: ID does not exist" containerID="50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0"
Oct 30 18:20:45.160590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.160531    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0"} err="failed to get container status \"50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0\": rpc error: code = NotFound desc = could not find container \"50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0\": container with ID starting with 50558bdbf4874986bcaca0fa13818b9569a61b034dde6f9e7e0d41f29de8e0b0 not found: ID does not exist"
Oct 30 18:20:45.200878 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.200847    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"]
Oct 30 18:20:45.212305 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:45.212256    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-57f9d9dd69-8wvgz"]
Oct 30 18:20:46.009802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:46.009765    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd86ddfd-979a-408f-827e-1c3b3463f84b" path="/var/lib/kubelet/pods/cd86ddfd-979a-408f-827e-1c3b3463f84b/volumes"
Oct 30 18:20:47.109314 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:47.109278    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:20:47.109706 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:47.109354    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:20:48.166924 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:48.166887    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-2" event={"ID":"f7f05e33-3be6-43f7-9696-09de0b1a8aa0","Type":"ContainerStarted","Data":"2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df"}
Oct 30 18:20:48.209770 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:48.209708    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-2" podStartSLOduration=1.921603391 podStartE2EDuration="6.209694504s" totalImagesPullingTime="4.288091113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:20:48.209155363 +0000 UTC m=+1418.375973503" watchObservedRunningTime="2026-10-30 18:20:48.209694504 +0000 UTC m=+1418.376512631"
Oct 30 18:20:59.179116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.179077    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-2"]
Oct 30 18:20:59.179624 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.179345    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-2" podUID="f7f05e33-3be6-43f7-9696-09de0b1a8aa0" containerName="otc-container" containerID="cri-o://2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df" gracePeriod=30
Oct 30 18:20:59.395416 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.395375    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"]
Oct 30 18:20:59.395680 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.395667    5746 cpu_manager.go:414] "containerMap: removing container" podUID="cd86ddfd-979a-408f-827e-1c3b3463f84b" containerName="ta-container"
Oct 30 18:20:59.395727 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.395680    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd86ddfd-979a-408f-827e-1c3b3463f84b" containerName="ta-container"
Oct 30 18:20:59.395761 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.395748    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd86ddfd-979a-408f-827e-1c3b3463f84b" containerName="ta-container"
Oct 30 18:20:59.398455 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.398436    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.419468 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.419447    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"]
Oct 30 18:20:59.500197 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.500127    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b8f8e866-7294-445d-aa75-9e17028dc33b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.500197 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.500171    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/b8f8e866-7294-445d-aa75-9e17028dc33b-ta-internal\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.500370 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.500257    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4h88\" (UniqueName: \"kubernetes.io/projected/b8f8e866-7294-445d-aa75-9e17028dc33b-kube-api-access-f4h88\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.601163 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.601130    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b8f8e866-7294-445d-aa75-9e17028dc33b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.601331 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.601174    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/b8f8e866-7294-445d-aa75-9e17028dc33b-ta-internal\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.601331 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.601200    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4h88\" (UniqueName: \"kubernetes.io/projected/b8f8e866-7294-445d-aa75-9e17028dc33b-kube-api-access-f4h88\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.601787 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.601758    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/b8f8e866-7294-445d-aa75-9e17028dc33b-ta-internal\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.603691 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.603667    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b8f8e866-7294-445d-aa75-9e17028dc33b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.623734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.623711    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4h88\" (UniqueName: \"kubernetes.io/projected/b8f8e866-7294-445d-aa75-9e17028dc33b-kube-api-access-f4h88\") pod \"data-science-collector-targetallocator-6f565b67cf-gndnb\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:20:59.709353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:20:59.709323    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:21:00.008420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.008391    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"]
Oct 30 18:21:00.010760 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:21:00.010732    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8f8e866_7294_445d_aa75_9e17028dc33b.slice/crio-bcea76fa0ecede4320e333302b3d3b12c2f78464bba1ab739557f69fd5a4b283 WatchSource:0}: Error finding container bcea76fa0ecede4320e333302b3d3b12c2f78464bba1ab739557f69fd5a4b283: Status 404 returned error can't find the container with id bcea76fa0ecede4320e333302b3d3b12c2f78464bba1ab739557f69fd5a4b283
Oct 30 18:21:00.012990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.012957    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:21:00.013094 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.013031    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:21:00.218042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.218004    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb" event={"ID":"b8f8e866-7294-445d-aa75-9e17028dc33b","Type":"ContainerStarted","Data":"94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908"}
Oct 30 18:21:00.218042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.218053    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb" event={"ID":"b8f8e866-7294-445d-aa75-9e17028dc33b","Type":"ContainerStarted","Data":"bcea76fa0ecede4320e333302b3d3b12c2f78464bba1ab739557f69fd5a4b283"}
Oct 30 18:21:00.218526 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.218074    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:21:00.219301 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.219279    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb" podUID="b8f8e866-7294-445d-aa75-9e17028dc33b" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.47:8080/readyz\": dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 18:21:00.261057 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.260881    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb" podStartSLOduration=1.260863958 podStartE2EDuration="1.260863958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:21:00.260788656 +0000 UTC m=+1430.427606822" watchObservedRunningTime="2026-10-30 18:21:00.260863958 +0000 UTC m=+1430.427682086"
Oct 30 18:21:00.413788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.413763    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:21:00.509008 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.508969    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otlp-tls-certs\") pod \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") "
Oct 30 18:21:00.509008 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.509014    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-kube-api-access-5g6hf\" (UniqueName: \"kubernetes.io/projected/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-kube-api-access-5g6hf\") pod \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") "
Oct 30 18:21:00.509256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.509049    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-data-science-collector-ta-client-cert\") pod \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") "
Oct 30 18:21:00.509256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.509170    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-monitoring-tls-certs\") pod \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") "
Oct 30 18:21:00.509256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.509219    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-tls-certs\") pod \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") "
Oct 30 18:21:00.509423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.509336    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otc-internal\" (UniqueName: \"kubernetes.io/configmap/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otc-internal\") pod \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\" (UID: \"f7f05e33-3be6-43f7-9696-09de0b1a8aa0\") "
Oct 30 18:21:00.509766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.509735    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otc-internal" pod "f7f05e33-3be6-43f7-9696-09de0b1a8aa0" (UID: "f7f05e33-3be6-43f7-9696-09de0b1a8aa0"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:21:00.511477 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.511424    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-tls-certs" pod "f7f05e33-3be6-43f7-9696-09de0b1a8aa0" (UID: "f7f05e33-3be6-43f7-9696-09de0b1a8aa0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:21:00.511843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.511825    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-data-science-collector-ta-client-cert" pod "f7f05e33-3be6-43f7-9696-09de0b1a8aa0" (UID: "f7f05e33-3be6-43f7-9696-09de0b1a8aa0"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:21:00.511898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.511838    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otlp-tls-certs" pod "f7f05e33-3be6-43f7-9696-09de0b1a8aa0" (UID: "f7f05e33-3be6-43f7-9696-09de0b1a8aa0"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:21:00.511991 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.511972    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-monitoring-tls-certs" pod "f7f05e33-3be6-43f7-9696-09de0b1a8aa0" (UID: "f7f05e33-3be6-43f7-9696-09de0b1a8aa0"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:21:00.512038 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.511993    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-kube-api-access-5g6hf" pod "f7f05e33-3be6-43f7-9696-09de0b1a8aa0" (UID: "f7f05e33-3be6-43f7-9696-09de0b1a8aa0"). InnerVolumeSpecName "kube-api-access-5g6hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:21:00.610319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.610293    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:00.610441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.610322    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:00.610441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.610336    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g6hf\" (UniqueName: \"kubernetes.io/projected/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-kube-api-access-5g6hf\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:00.610441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.610349    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:00.610441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.610362    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:00.610441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:00.610374    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f05e33-3be6-43f7-9696-09de0b1a8aa0-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:01.226412 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.226373    5746 generic.go:356] "Generic (PLEG): container finished" podID="f7f05e33-3be6-43f7-9696-09de0b1a8aa0" containerID="2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df" exitCode=0
Oct 30 18:21:01.226902 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.226449    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-2"
Oct 30 18:21:01.226902 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.226526    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-2" event={"ID":"f7f05e33-3be6-43f7-9696-09de0b1a8aa0","Type":"ContainerDied","Data":"2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df"}
Oct 30 18:21:01.226902 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.226580    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-2" event={"ID":"f7f05e33-3be6-43f7-9696-09de0b1a8aa0","Type":"ContainerDied","Data":"062ad6adb02fe4a0ec8b63214c38b2d016345a653f9cd30575c157ac1cddd55b"}
Oct 30 18:21:01.226902 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.226654    5746 scope.go:122] "RemoveContainer" containerID="2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df"
Oct 30 18:21:01.232531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.232509    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:21:01.241211 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.241190    5746 scope.go:122] "RemoveContainer" containerID="2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df"
Oct 30 18:21:01.241962 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:21:01.241933    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df\": container with ID starting with 2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df not found: ID does not exist" containerID="2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df"
Oct 30 18:21:01.242085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.241973    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df"} err="failed to get container status \"2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df\": rpc error: code = NotFound desc = could not find container \"2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df\": container with ID starting with 2a690758cc681b9cece157a2b833d80cfb6b3554b3ff7570d90648a96bab74df not found: ID does not exist"
Oct 30 18:21:01.278229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.278199    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-2"]
Oct 30 18:21:01.290126 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.290088    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-2"]
Oct 30 18:21:01.357836 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.357807    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"]
Oct 30 18:21:01.358031 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.358009    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m" podUID="e1d550bd-15e0-49f9-9c0b-556d480de831" containerName="ta-container" containerID="cri-o://c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df" gracePeriod=30
Oct 30 18:21:01.602666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.602635    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:21:01.718997 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.718961    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1d550bd-15e0-49f9-9c0b-556d480de831-ta-internal\" (UniqueName: \"kubernetes.io/configmap/e1d550bd-15e0-49f9-9c0b-556d480de831-ta-internal\") pod \"e1d550bd-15e0-49f9-9c0b-556d480de831\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") "
Oct 30 18:21:01.719205 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.719051    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1d550bd-15e0-49f9-9c0b-556d480de831-kube-api-access-mmj9z\" (UniqueName: \"kubernetes.io/projected/e1d550bd-15e0-49f9-9c0b-556d480de831-kube-api-access-mmj9z\") pod \"e1d550bd-15e0-49f9-9c0b-556d480de831\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") "
Oct 30 18:21:01.719205 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.719082    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1d550bd-15e0-49f9-9c0b-556d480de831-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e1d550bd-15e0-49f9-9c0b-556d480de831-data-science-collector-ta-server-cert\") pod \"e1d550bd-15e0-49f9-9c0b-556d480de831\" (UID: \"e1d550bd-15e0-49f9-9c0b-556d480de831\") "
Oct 30 18:21:01.719332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.719294    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1d550bd-15e0-49f9-9c0b-556d480de831-ta-internal" pod "e1d550bd-15e0-49f9-9c0b-556d480de831" (UID: "e1d550bd-15e0-49f9-9c0b-556d480de831"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:21:01.721333 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.721303    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1d550bd-15e0-49f9-9c0b-556d480de831-kube-api-access-mmj9z" pod "e1d550bd-15e0-49f9-9c0b-556d480de831" (UID: "e1d550bd-15e0-49f9-9c0b-556d480de831"). InnerVolumeSpecName "kube-api-access-mmj9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:21:01.721440 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.721352    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1d550bd-15e0-49f9-9c0b-556d480de831-data-science-collector-ta-server-cert" pod "e1d550bd-15e0-49f9-9c0b-556d480de831" (UID: "e1d550bd-15e0-49f9-9c0b-556d480de831"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:21:01.819618 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.819542    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/e1d550bd-15e0-49f9-9c0b-556d480de831-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:01.819618 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.819568    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmj9z\" (UniqueName: \"kubernetes.io/projected/e1d550bd-15e0-49f9-9c0b-556d480de831-kube-api-access-mmj9z\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:01.819618 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:01.819578    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e1d550bd-15e0-49f9-9c0b-556d480de831-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:02.008329 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.008293    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7f05e33-3be6-43f7-9696-09de0b1a8aa0" path="/var/lib/kubelet/pods/f7f05e33-3be6-43f7-9696-09de0b1a8aa0/volumes"
Oct 30 18:21:02.232014 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.231974    5746 generic.go:356] "Generic (PLEG): container finished" podID="e1d550bd-15e0-49f9-9c0b-556d480de831" containerID="c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df" exitCode=0
Oct 30 18:21:02.232497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.232054    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"
Oct 30 18:21:02.232497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.232054    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m" event={"ID":"e1d550bd-15e0-49f9-9c0b-556d480de831","Type":"ContainerDied","Data":"c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df"}
Oct 30 18:21:02.232497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.232173    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m" event={"ID":"e1d550bd-15e0-49f9-9c0b-556d480de831","Type":"ContainerDied","Data":"a19f0cb137d387b566f78b3aeea23d15a5337856127cf5d925ecfce05a91ba30"}
Oct 30 18:21:02.232497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.232195    5746 scope.go:122] "RemoveContainer" containerID="c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df"
Oct 30 18:21:02.240499 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.240480    5746 scope.go:122] "RemoveContainer" containerID="c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df"
Oct 30 18:21:02.240769 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:21:02.240748    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df\": container with ID starting with c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df not found: ID does not exist" containerID="c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df"
Oct 30 18:21:02.240852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.240776    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df"} err="failed to get container status \"c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df\": rpc error: code = NotFound desc = could not find container \"c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df\": container with ID starting with c42e6b4bfaacd0535deb9e5ba48bcba63d65df4efaa5fdb7ad4cdc5465c726df not found: ID does not exist"
Oct 30 18:21:02.281474 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.281442    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"]
Oct 30 18:21:02.285340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:02.285316    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6db65bc7b6-7pp4m"]
Oct 30 18:21:04.008760 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:04.008726    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1d550bd-15e0-49f9-9c0b-556d480de831" path="/var/lib/kubelet/pods/e1d550bd-15e0-49f9-9c0b-556d480de831/volumes"
Oct 30 18:21:06.906055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:06.906029    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"]
Oct 30 18:21:06.906449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:06.906316    5746 cpu_manager.go:414] "containerMap: removing container" podUID="e1d550bd-15e0-49f9-9c0b-556d480de831" containerName="ta-container"
Oct 30 18:21:06.906449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:06.906327    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1d550bd-15e0-49f9-9c0b-556d480de831" containerName="ta-container"
Oct 30 18:21:06.906449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:06.906346    5746 cpu_manager.go:414] "containerMap: removing container" podUID="f7f05e33-3be6-43f7-9696-09de0b1a8aa0" containerName="otc-container"
Oct 30 18:21:06.906449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:06.906351    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7f05e33-3be6-43f7-9696-09de0b1a8aa0" containerName="otc-container"
Oct 30 18:21:06.906449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:06.906402    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7f05e33-3be6-43f7-9696-09de0b1a8aa0" containerName="otc-container"
Oct 30 18:21:06.906449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:06.906414    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1d550bd-15e0-49f9-9c0b-556d480de831" containerName="ta-container"
Oct 30 18:21:06.910254 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:06.910190    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.062455 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.062424    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fwzv\" (UniqueName: \"kubernetes.io/projected/8d68d18d-6c9a-4711-9151-02be0b14bcad-kube-api-access-9fwzv\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.062620 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.062482    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/8d68d18d-6c9a-4711-9151-02be0b14bcad-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.062620 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.062580    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/8d68d18d-6c9a-4711-9151-02be0b14bcad-ta-internal\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.163621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.163534    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/8d68d18d-6c9a-4711-9151-02be0b14bcad-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.163621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.163585    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/8d68d18d-6c9a-4711-9151-02be0b14bcad-ta-internal\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.163621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.163617    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fwzv\" (UniqueName: \"kubernetes.io/projected/8d68d18d-6c9a-4711-9151-02be0b14bcad-kube-api-access-9fwzv\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.164300 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.164227    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/8d68d18d-6c9a-4711-9151-02be0b14bcad-ta-internal\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.166134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.166113    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/8d68d18d-6c9a-4711-9151-02be0b14bcad-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.186868 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.186845    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fwzv\" (UniqueName: \"kubernetes.io/projected/8d68d18d-6c9a-4711-9151-02be0b14bcad-kube-api-access-9fwzv\") pod \"data-science-collector-targetallocator-86869b46c7-b9sbb\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:07.221823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:07.221802    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:12.790124 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:21:12.790048    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:21:14.734437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:14.734399    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"]
Oct 30 18:21:14.783575 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:14.783542    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"]
Oct 30 18:21:14.788156 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:14.788138    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:14.928859 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:14.928826    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vslwd\" (UniqueName: \"kubernetes.io/projected/1fa58d6a-4173-4212-a49f-2d7192b0b185-kube-api-access-vslwd\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:14.929024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:14.928872    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1fa58d6a-4173-4212-a49f-2d7192b0b185-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:14.929024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:14.928953    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/1fa58d6a-4173-4212-a49f-2d7192b0b185-ta-internal\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:15.029376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:15.029288    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1fa58d6a-4173-4212-a49f-2d7192b0b185-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:15.029376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:15.029340    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/1fa58d6a-4173-4212-a49f-2d7192b0b185-ta-internal\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:15.029604 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:15.029400    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vslwd\" (UniqueName: \"kubernetes.io/projected/1fa58d6a-4173-4212-a49f-2d7192b0b185-kube-api-access-vslwd\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:15.030015 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:15.029995    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/1fa58d6a-4173-4212-a49f-2d7192b0b185-ta-internal\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:15.031868 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:15.031849    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1fa58d6a-4173-4212-a49f-2d7192b0b185-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:15.052248 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:15.052227    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vslwd\" (UniqueName: \"kubernetes.io/projected/1fa58d6a-4173-4212-a49f-2d7192b0b185-kube-api-access-vslwd\") pod \"data-science-collector-targetallocator-7c98f9747f-7fh2x\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:15.099190 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:15.099164    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:42.790196 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:21:42.790116    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:21:48.751537 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:48.751497    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"]
Oct 30 18:21:48.762893 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:48.762865    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"]
Oct 30 18:21:49.217362 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.217334    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"]
Oct 30 18:21:49.220756 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:21:49.220726    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fa58d6a_4173_4212_a49f_2d7192b0b185.slice/crio-706559e46d42fe6d741e06d683b2f41bd210fa7edd36aaaeb98d738c98991816 WatchSource:0}: Error finding container 706559e46d42fe6d741e06d683b2f41bd210fa7edd36aaaeb98d738c98991816: Status 404 returned error can't find the container with id 706559e46d42fe6d741e06d683b2f41bd210fa7edd36aaaeb98d738c98991816
Oct 30 18:21:49.223338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.223302    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:21:49.223422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.223360    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:21:49.223864 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:21:49.223846    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d68d18d_6c9a_4711_9151_02be0b14bcad.slice/crio-ee633dbc976f8db0eb905c199cad5858ae83fce927c82275de964afd989e3e7f WatchSource:0}: Error finding container ee633dbc976f8db0eb905c199cad5858ae83fce927c82275de964afd989e3e7f: Status 404 returned error can't find the container with id ee633dbc976f8db0eb905c199cad5858ae83fce927c82275de964afd989e3e7f
Oct 30 18:21:49.228830 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.228809    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"]
Oct 30 18:21:49.229398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.229367    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:21:49.229490 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.229421    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:21:49.417950 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.417893    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x" event={"ID":"1fa58d6a-4173-4212-a49f-2d7192b0b185","Type":"ContainerStarted","Data":"89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140"}
Oct 30 18:21:49.418170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.417958    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x" event={"ID":"1fa58d6a-4173-4212-a49f-2d7192b0b185","Type":"ContainerStarted","Data":"706559e46d42fe6d741e06d683b2f41bd210fa7edd36aaaeb98d738c98991816"}
Oct 30 18:21:49.418170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.417982    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:49.419349 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.419317    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x" podUID="1fa58d6a-4173-4212-a49f-2d7192b0b185" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.48:8080/readyz\": dial tcp 10.133.0.48:8080: connect: connection refused"
Oct 30 18:21:49.419854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.419836    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb" event={"ID":"8d68d18d-6c9a-4711-9151-02be0b14bcad","Type":"ContainerStarted","Data":"acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075"}
Oct 30 18:21:49.419946 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.419867    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb" event={"ID":"8d68d18d-6c9a-4711-9151-02be0b14bcad","Type":"ContainerStarted","Data":"ee633dbc976f8db0eb905c199cad5858ae83fce927c82275de964afd989e3e7f"}
Oct 30 18:21:49.419946 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.419901    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb" podUID="8d68d18d-6c9a-4711-9151-02be0b14bcad" containerName="ta-container" containerID="cri-o://acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075" gracePeriod=30
Oct 30 18:21:49.420054 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.419965    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:49.420981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.420957    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb" podUID="8d68d18d-6c9a-4711-9151-02be0b14bcad" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.49:8080/readyz\": dial tcp 10.133.0.49:8080: connect: connection refused"
Oct 30 18:21:49.459937 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.459780    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x" podStartSLOduration=35.459765413 podStartE2EDuration="35.459765413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:21:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:21:49.459178514 +0000 UTC m=+1479.625996666" watchObservedRunningTime="2026-10-30 18:21:49.459765413 +0000 UTC m=+1479.626583541"
Oct 30 18:21:49.500332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.500252    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb" podStartSLOduration=43.500229582 podStartE2EDuration="43.500229582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:21:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:21:49.499712814 +0000 UTC m=+1479.666530943" watchObservedRunningTime="2026-10-30 18:21:49.500229582 +0000 UTC m=+1479.667047706"
Oct 30 18:21:49.751776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.751723    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:49.800526 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.800499    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d68d18d-6c9a-4711-9151-02be0b14bcad-kube-api-access-9fwzv\" (UniqueName: \"kubernetes.io/projected/8d68d18d-6c9a-4711-9151-02be0b14bcad-kube-api-access-9fwzv\") pod \"8d68d18d-6c9a-4711-9151-02be0b14bcad\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") "
Oct 30 18:21:49.800666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.800555    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d68d18d-6c9a-4711-9151-02be0b14bcad-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/8d68d18d-6c9a-4711-9151-02be0b14bcad-data-science-collector-ta-server-cert\") pod \"8d68d18d-6c9a-4711-9151-02be0b14bcad\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") "
Oct 30 18:21:49.800666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.800580    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d68d18d-6c9a-4711-9151-02be0b14bcad-ta-internal\" (UniqueName: \"kubernetes.io/configmap/8d68d18d-6c9a-4711-9151-02be0b14bcad-ta-internal\") pod \"8d68d18d-6c9a-4711-9151-02be0b14bcad\" (UID: \"8d68d18d-6c9a-4711-9151-02be0b14bcad\") "
Oct 30 18:21:49.801019 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.800988    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d68d18d-6c9a-4711-9151-02be0b14bcad-ta-internal" pod "8d68d18d-6c9a-4711-9151-02be0b14bcad" (UID: "8d68d18d-6c9a-4711-9151-02be0b14bcad"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:21:49.802670 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.802641    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d68d18d-6c9a-4711-9151-02be0b14bcad-kube-api-access-9fwzv" pod "8d68d18d-6c9a-4711-9151-02be0b14bcad" (UID: "8d68d18d-6c9a-4711-9151-02be0b14bcad"). InnerVolumeSpecName "kube-api-access-9fwzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:21:49.802779 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.802754    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d68d18d-6c9a-4711-9151-02be0b14bcad-data-science-collector-ta-server-cert" pod "8d68d18d-6c9a-4711-9151-02be0b14bcad" (UID: "8d68d18d-6c9a-4711-9151-02be0b14bcad"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:21:49.901639 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.901602    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fwzv\" (UniqueName: \"kubernetes.io/projected/8d68d18d-6c9a-4711-9151-02be0b14bcad-kube-api-access-9fwzv\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:49.901639 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.901635    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/8d68d18d-6c9a-4711-9151-02be0b14bcad-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:49.901834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:49.901652    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/8d68d18d-6c9a-4711-9151-02be0b14bcad-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:50.425023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.424992    5746 generic.go:356] "Generic (PLEG): container finished" podID="8d68d18d-6c9a-4711-9151-02be0b14bcad" containerID="acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075" exitCode=0
Oct 30 18:21:50.425196 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.425037    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"
Oct 30 18:21:50.425196 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.425087    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb" event={"ID":"8d68d18d-6c9a-4711-9151-02be0b14bcad","Type":"ContainerDied","Data":"acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075"}
Oct 30 18:21:50.425196 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.425134    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb" event={"ID":"8d68d18d-6c9a-4711-9151-02be0b14bcad","Type":"ContainerDied","Data":"ee633dbc976f8db0eb905c199cad5858ae83fce927c82275de964afd989e3e7f"}
Oct 30 18:21:50.425196 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.425151    5746 scope.go:122] "RemoveContainer" containerID="acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075"
Oct 30 18:21:50.426795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.426775    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:21:50.433415 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.433399    5746 scope.go:122] "RemoveContainer" containerID="acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075"
Oct 30 18:21:50.433691 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:21:50.433673    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075\": container with ID starting with acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075 not found: ID does not exist" containerID="acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075"
Oct 30 18:21:50.433758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.433699    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075"} err="failed to get container status \"acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075\": rpc error: code = NotFound desc = could not find container \"acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075\": container with ID starting with acdc925a8de891cdd8922f6867a957489c9da3303e4efe36ad02e8f367e9f075 not found: ID does not exist"
Oct 30 18:21:50.508835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.508807    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"]
Oct 30 18:21:50.509070 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.509047    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb" podUID="b8f8e866-7294-445d-aa75-9e17028dc33b" containerName="ta-container" containerID="cri-o://94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908" gracePeriod=30
Oct 30 18:21:50.518360 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.518333    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"]
Oct 30 18:21:50.521590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.521560    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-86869b46c7-b9sbb"]
Oct 30 18:21:50.747512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.747480    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:21:50.807365 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.807338    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8f8e866-7294-445d-aa75-9e17028dc33b-kube-api-access-f4h88\" (UniqueName: \"kubernetes.io/projected/b8f8e866-7294-445d-aa75-9e17028dc33b-kube-api-access-f4h88\") pod \"b8f8e866-7294-445d-aa75-9e17028dc33b\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") "
Oct 30 18:21:50.807820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.807390    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8f8e866-7294-445d-aa75-9e17028dc33b-ta-internal\" (UniqueName: \"kubernetes.io/configmap/b8f8e866-7294-445d-aa75-9e17028dc33b-ta-internal\") pod \"b8f8e866-7294-445d-aa75-9e17028dc33b\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") "
Oct 30 18:21:50.807820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.807431    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8f8e866-7294-445d-aa75-9e17028dc33b-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b8f8e866-7294-445d-aa75-9e17028dc33b-data-science-collector-ta-server-cert\") pod \"b8f8e866-7294-445d-aa75-9e17028dc33b\" (UID: \"b8f8e866-7294-445d-aa75-9e17028dc33b\") "
Oct 30 18:21:50.807820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.807777    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f8e866-7294-445d-aa75-9e17028dc33b-ta-internal" pod "b8f8e866-7294-445d-aa75-9e17028dc33b" (UID: "b8f8e866-7294-445d-aa75-9e17028dc33b"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:21:50.809721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.809696    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8f8e866-7294-445d-aa75-9e17028dc33b-kube-api-access-f4h88" pod "b8f8e866-7294-445d-aa75-9e17028dc33b" (UID: "b8f8e866-7294-445d-aa75-9e17028dc33b"). InnerVolumeSpecName "kube-api-access-f4h88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:21:50.809721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.809703    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8f8e866-7294-445d-aa75-9e17028dc33b-data-science-collector-ta-server-cert" pod "b8f8e866-7294-445d-aa75-9e17028dc33b" (UID: "b8f8e866-7294-445d-aa75-9e17028dc33b"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:21:50.908448 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.908416    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4h88\" (UniqueName: \"kubernetes.io/projected/b8f8e866-7294-445d-aa75-9e17028dc33b-kube-api-access-f4h88\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:50.908448 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.908448    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/b8f8e866-7294-445d-aa75-9e17028dc33b-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:50.908641 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:50.908464    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/b8f8e866-7294-445d-aa75-9e17028dc33b-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:21:51.431005 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.430974    5746 generic.go:356] "Generic (PLEG): container finished" podID="b8f8e866-7294-445d-aa75-9e17028dc33b" containerID="94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908" exitCode=0
Oct 30 18:21:51.431213 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.431031    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"
Oct 30 18:21:51.431213 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.431066    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb" event={"ID":"b8f8e866-7294-445d-aa75-9e17028dc33b","Type":"ContainerDied","Data":"94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908"}
Oct 30 18:21:51.431213 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.431109    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb" event={"ID":"b8f8e866-7294-445d-aa75-9e17028dc33b","Type":"ContainerDied","Data":"bcea76fa0ecede4320e333302b3d3b12c2f78464bba1ab739557f69fd5a4b283"}
Oct 30 18:21:51.431213 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.431126    5746 scope.go:122] "RemoveContainer" containerID="94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908"
Oct 30 18:21:51.440010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.439993    5746 scope.go:122] "RemoveContainer" containerID="94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908"
Oct 30 18:21:51.440267 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:21:51.440248    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908\": container with ID starting with 94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908 not found: ID does not exist" containerID="94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908"
Oct 30 18:21:51.440331 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.440275    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908"} err="failed to get container status \"94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908\": rpc error: code = NotFound desc = could not find container \"94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908\": container with ID starting with 94cb3530c59dd299d29af3efe2c734a9ebb20ca1664ed5ac62e17395bf58b908 not found: ID does not exist"
Oct 30 18:21:51.481131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.481107    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"]
Oct 30 18:21:51.485411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:51.485392    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-gndnb"]
Oct 30 18:21:52.008143 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:52.008113    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d68d18d-6c9a-4711-9151-02be0b14bcad" path="/var/lib/kubelet/pods/8d68d18d-6c9a-4711-9151-02be0b14bcad/volumes"
Oct 30 18:21:52.008526 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:21:52.008463    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8f8e866-7294-445d-aa75-9e17028dc33b" path="/var/lib/kubelet/pods/b8f8e866-7294-445d-aa75-9e17028dc33b/volumes"
Oct 30 18:22:12.790123 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:22:12.790088    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:22:42.789499 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:22:42.789453    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:23:12.789841 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:23:12.789802    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:23:27.433238 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.433205    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"]
Oct 30 18:23:27.433866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.433644    5746 cpu_manager.go:414] "containerMap: removing container" podUID="8d68d18d-6c9a-4711-9151-02be0b14bcad" containerName="ta-container"
Oct 30 18:23:27.433866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.433662    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d68d18d-6c9a-4711-9151-02be0b14bcad" containerName="ta-container"
Oct 30 18:23:27.433866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.433700    5746 cpu_manager.go:414] "containerMap: removing container" podUID="b8f8e866-7294-445d-aa75-9e17028dc33b" containerName="ta-container"
Oct 30 18:23:27.433866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.433708    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8f8e866-7294-445d-aa75-9e17028dc33b" containerName="ta-container"
Oct 30 18:23:27.433866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.433789    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d68d18d-6c9a-4711-9151-02be0b14bcad" containerName="ta-container"
Oct 30 18:23:27.433866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.433804    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8f8e866-7294-445d-aa75-9e17028dc33b" containerName="ta-container"
Oct 30 18:23:27.436741 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.436720    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.476742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.476716    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"]
Oct 30 18:23:27.497532 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.497496    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmsn6\" (UniqueName: \"kubernetes.io/projected/ef99c56c-d2e7-420c-9d5e-22763166df9f-kube-api-access-rmsn6\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.497658 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.497538    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ef99c56c-d2e7-420c-9d5e-22763166df9f-ta-internal\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.497658 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.497575    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ef99c56c-d2e7-420c-9d5e-22763166df9f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.598067 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.598035    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmsn6\" (UniqueName: \"kubernetes.io/projected/ef99c56c-d2e7-420c-9d5e-22763166df9f-kube-api-access-rmsn6\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.598067 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.598068    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ef99c56c-d2e7-420c-9d5e-22763166df9f-ta-internal\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.598236 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.598090    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ef99c56c-d2e7-420c-9d5e-22763166df9f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.598787 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.598765    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ef99c56c-d2e7-420c-9d5e-22763166df9f-ta-internal\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.600734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.600712    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ef99c56c-d2e7-420c-9d5e-22763166df9f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.621398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.621369    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmsn6\" (UniqueName: \"kubernetes.io/projected/ef99c56c-d2e7-420c-9d5e-22763166df9f-kube-api-access-rmsn6\") pod \"data-science-collector-targetallocator-6f565b67cf-vsfzv\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:27.747502 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:27.747427    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:28.042784 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.042682    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"]
Oct 30 18:23:28.046040 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:23:28.046009    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef99c56c_d2e7_420c_9d5e_22763166df9f.slice/crio-f45d4a00a788407935e6c83243fffc908507cb6f1b9bb20c11939160a1f5dd3b WatchSource:0}: Error finding container f45d4a00a788407935e6c83243fffc908507cb6f1b9bb20c11939160a1f5dd3b: Status 404 returned error can't find the container with id f45d4a00a788407935e6c83243fffc908507cb6f1b9bb20c11939160a1f5dd3b
Oct 30 18:23:28.048210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.048171    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:23:28.048292 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.048245    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:23:28.797646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.797614    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv" event={"ID":"ef99c56c-d2e7-420c-9d5e-22763166df9f","Type":"ContainerStarted","Data":"59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432"}
Oct 30 18:23:28.797646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.797652    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv" event={"ID":"ef99c56c-d2e7-420c-9d5e-22763166df9f","Type":"ContainerStarted","Data":"f45d4a00a788407935e6c83243fffc908507cb6f1b9bb20c11939160a1f5dd3b"}
Oct 30 18:23:28.798142 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.797800    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:28.799578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.799554    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:28.838896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.838844    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv" podStartSLOduration=1.8388329300000001 podStartE2EDuration="1.83883293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:23:28.838126994 +0000 UTC m=+1579.004945120" watchObservedRunningTime="2026-10-30 18:23:28.83883293 +0000 UTC m=+1579.005651081"
Oct 30 18:23:28.910693 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.910662    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"]
Oct 30 18:23:28.910890 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:28.910866    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x" podUID="1fa58d6a-4173-4212-a49f-2d7192b0b185" containerName="ta-container" containerID="cri-o://89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140" gracePeriod=30
Oct 30 18:23:29.151224 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.151195    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:23:29.212164 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.212137    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fa58d6a-4173-4212-a49f-2d7192b0b185-kube-api-access-vslwd\" (UniqueName: \"kubernetes.io/projected/1fa58d6a-4173-4212-a49f-2d7192b0b185-kube-api-access-vslwd\") pod \"1fa58d6a-4173-4212-a49f-2d7192b0b185\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") "
Oct 30 18:23:29.212312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.212186    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fa58d6a-4173-4212-a49f-2d7192b0b185-ta-internal\" (UniqueName: \"kubernetes.io/configmap/1fa58d6a-4173-4212-a49f-2d7192b0b185-ta-internal\") pod \"1fa58d6a-4173-4212-a49f-2d7192b0b185\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") "
Oct 30 18:23:29.212312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.212217    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1fa58d6a-4173-4212-a49f-2d7192b0b185-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1fa58d6a-4173-4212-a49f-2d7192b0b185-data-science-collector-ta-server-cert\") pod \"1fa58d6a-4173-4212-a49f-2d7192b0b185\" (UID: \"1fa58d6a-4173-4212-a49f-2d7192b0b185\") "
Oct 30 18:23:29.212546 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.212524    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fa58d6a-4173-4212-a49f-2d7192b0b185-ta-internal" pod "1fa58d6a-4173-4212-a49f-2d7192b0b185" (UID: "1fa58d6a-4173-4212-a49f-2d7192b0b185"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:23:29.214463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.214434    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa58d6a-4173-4212-a49f-2d7192b0b185-kube-api-access-vslwd" pod "1fa58d6a-4173-4212-a49f-2d7192b0b185" (UID: "1fa58d6a-4173-4212-a49f-2d7192b0b185"). InnerVolumeSpecName "kube-api-access-vslwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:23:29.214463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.214449    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa58d6a-4173-4212-a49f-2d7192b0b185-data-science-collector-ta-server-cert" pod "1fa58d6a-4173-4212-a49f-2d7192b0b185" (UID: "1fa58d6a-4173-4212-a49f-2d7192b0b185"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:23:29.313482 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.313450    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1fa58d6a-4173-4212-a49f-2d7192b0b185-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:29.313482 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.313479    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vslwd\" (UniqueName: \"kubernetes.io/projected/1fa58d6a-4173-4212-a49f-2d7192b0b185-kube-api-access-vslwd\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:29.313482 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.313489    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/1fa58d6a-4173-4212-a49f-2d7192b0b185-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:29.803161 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.803123    5746 generic.go:356] "Generic (PLEG): container finished" podID="1fa58d6a-4173-4212-a49f-2d7192b0b185" containerID="89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140" exitCode=0
Oct 30 18:23:29.803633 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.803169    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"
Oct 30 18:23:29.803633 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.803210    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x" event={"ID":"1fa58d6a-4173-4212-a49f-2d7192b0b185","Type":"ContainerDied","Data":"89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140"}
Oct 30 18:23:29.803633 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.803248    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x" event={"ID":"1fa58d6a-4173-4212-a49f-2d7192b0b185","Type":"ContainerDied","Data":"706559e46d42fe6d741e06d683b2f41bd210fa7edd36aaaeb98d738c98991816"}
Oct 30 18:23:29.803633 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.803271    5746 scope.go:122] "RemoveContainer" containerID="89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140"
Oct 30 18:23:29.812034 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.812019    5746 scope.go:122] "RemoveContainer" containerID="89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140"
Oct 30 18:23:29.812329 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:23:29.812312    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140\": container with ID starting with 89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140 not found: ID does not exist" containerID="89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140"
Oct 30 18:23:29.812374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.812339    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140"} err="failed to get container status \"89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140\": rpc error: code = NotFound desc = could not find container \"89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140\": container with ID starting with 89f2cbe3bc7e34de706c90b31eb3d8404fc47a1094a16e75312df0a86f2df140 not found: ID does not exist"
Oct 30 18:23:29.852604 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.852581    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"]
Oct 30 18:23:29.864832 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:29.864811    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7c98f9747f-7fh2x"]
Oct 30 18:23:30.007842 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:30.007810    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fa58d6a-4173-4212-a49f-2d7192b0b185" path="/var/lib/kubelet/pods/1fa58d6a-4173-4212-a49f-2d7192b0b185/volumes"
Oct 30 18:23:36.890201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.890169    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"]
Oct 30 18:23:36.890581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.890445    5746 cpu_manager.go:414] "containerMap: removing container" podUID="1fa58d6a-4173-4212-a49f-2d7192b0b185" containerName="ta-container"
Oct 30 18:23:36.890581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.890454    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fa58d6a-4173-4212-a49f-2d7192b0b185" containerName="ta-container"
Oct 30 18:23:36.890581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.890502    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fa58d6a-4173-4212-a49f-2d7192b0b185" containerName="ta-container"
Oct 30 18:23:36.894747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.894731    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:36.922860 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.922822    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"]
Oct 30 18:23:36.976457 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.976426    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gcz6\" (UniqueName: \"kubernetes.io/projected/553b242b-afca-46f1-8793-3df008282823-kube-api-access-9gcz6\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:36.976599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.976473    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/553b242b-afca-46f1-8793-3df008282823-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:36.976599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:36.976555    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/553b242b-afca-46f1-8793-3df008282823-ta-internal\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.077344 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.077314    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gcz6\" (UniqueName: \"kubernetes.io/projected/553b242b-afca-46f1-8793-3df008282823-kube-api-access-9gcz6\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.077495 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.077364    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/553b242b-afca-46f1-8793-3df008282823-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.077495 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.077398    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/553b242b-afca-46f1-8793-3df008282823-ta-internal\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.078087 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.078069    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/553b242b-afca-46f1-8793-3df008282823-ta-internal\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.080006 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.079987    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/553b242b-afca-46f1-8793-3df008282823-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.100277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.100254    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gcz6\" (UniqueName: \"kubernetes.io/projected/553b242b-afca-46f1-8793-3df008282823-kube-api-access-9gcz6\") pod \"data-science-collector-targetallocator-cd46d58f7-88smz\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.205998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.205924    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.508127 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.508050    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"]
Oct 30 18:23:37.511284 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:23:37.511255    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod553b242b_afca_46f1_8793_3df008282823.slice/crio-69ae2b6aaee5a05311446ff89d003281c0b699a397dd1300d8695d922618b3f7 WatchSource:0}: Error finding container 69ae2b6aaee5a05311446ff89d003281c0b699a397dd1300d8695d922618b3f7: Status 404 returned error can't find the container with id 69ae2b6aaee5a05311446ff89d003281c0b699a397dd1300d8695d922618b3f7
Oct 30 18:23:37.513341 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.513309    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:23:37.513445 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.513378    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:23:37.837010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.836898    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz" event={"ID":"553b242b-afca-46f1-8793-3df008282823","Type":"ContainerStarted","Data":"28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075"}
Oct 30 18:23:37.837010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.836966    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz" event={"ID":"553b242b-afca-46f1-8793-3df008282823","Type":"ContainerStarted","Data":"69ae2b6aaee5a05311446ff89d003281c0b699a397dd1300d8695d922618b3f7"}
Oct 30 18:23:37.837204 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.837074    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:37.838044 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.838018    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz" podUID="553b242b-afca-46f1-8793-3df008282823" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.51:8080/readyz\": dial tcp 10.133.0.51:8080: connect: connection refused"
Oct 30 18:23:37.878277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:37.878232    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz" podStartSLOduration=1.878220392 podStartE2EDuration="1.878220392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:23:37.877445796 +0000 UTC m=+1588.044263939" watchObservedRunningTime="2026-10-30 18:23:37.878220392 +0000 UTC m=+1588.045038520"
Oct 30 18:23:38.841780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:38.841754    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:38.924276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:38.924245    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"]
Oct 30 18:23:38.924473 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:38.924445    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv" podUID="ef99c56c-d2e7-420c-9d5e-22763166df9f" containerName="ta-container" containerID="cri-o://59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432" gracePeriod=30
Oct 30 18:23:39.163029 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.163001    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:39.295501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.295470    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef99c56c-d2e7-420c-9d5e-22763166df9f-ta-internal\" (UniqueName: \"kubernetes.io/configmap/ef99c56c-d2e7-420c-9d5e-22763166df9f-ta-internal\") pod \"ef99c56c-d2e7-420c-9d5e-22763166df9f\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") "
Oct 30 18:23:39.295699 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.295567    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef99c56c-d2e7-420c-9d5e-22763166df9f-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ef99c56c-d2e7-420c-9d5e-22763166df9f-data-science-collector-ta-server-cert\") pod \"ef99c56c-d2e7-420c-9d5e-22763166df9f\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") "
Oct 30 18:23:39.295699 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.295596    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef99c56c-d2e7-420c-9d5e-22763166df9f-kube-api-access-rmsn6\" (UniqueName: \"kubernetes.io/projected/ef99c56c-d2e7-420c-9d5e-22763166df9f-kube-api-access-rmsn6\") pod \"ef99c56c-d2e7-420c-9d5e-22763166df9f\" (UID: \"ef99c56c-d2e7-420c-9d5e-22763166df9f\") "
Oct 30 18:23:39.295896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.295869    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef99c56c-d2e7-420c-9d5e-22763166df9f-ta-internal" pod "ef99c56c-d2e7-420c-9d5e-22763166df9f" (UID: "ef99c56c-d2e7-420c-9d5e-22763166df9f"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:23:39.297711 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.297690    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef99c56c-d2e7-420c-9d5e-22763166df9f-data-science-collector-ta-server-cert" pod "ef99c56c-d2e7-420c-9d5e-22763166df9f" (UID: "ef99c56c-d2e7-420c-9d5e-22763166df9f"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:23:39.297819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.297765    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef99c56c-d2e7-420c-9d5e-22763166df9f-kube-api-access-rmsn6" pod "ef99c56c-d2e7-420c-9d5e-22763166df9f" (UID: "ef99c56c-d2e7-420c-9d5e-22763166df9f"). InnerVolumeSpecName "kube-api-access-rmsn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:23:39.396549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.396522    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ef99c56c-d2e7-420c-9d5e-22763166df9f-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:39.396549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.396550    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ef99c56c-d2e7-420c-9d5e-22763166df9f-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:39.396758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.396561    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmsn6\" (UniqueName: \"kubernetes.io/projected/ef99c56c-d2e7-420c-9d5e-22763166df9f-kube-api-access-rmsn6\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:39.845791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.845707    5746 generic.go:356] "Generic (PLEG): container finished" podID="ef99c56c-d2e7-420c-9d5e-22763166df9f" containerID="59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432" exitCode=0
Oct 30 18:23:39.845791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.845756    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"
Oct 30 18:23:39.846284 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.845796    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv" event={"ID":"ef99c56c-d2e7-420c-9d5e-22763166df9f","Type":"ContainerDied","Data":"59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432"}
Oct 30 18:23:39.846284 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.845833    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv" event={"ID":"ef99c56c-d2e7-420c-9d5e-22763166df9f","Type":"ContainerDied","Data":"f45d4a00a788407935e6c83243fffc908507cb6f1b9bb20c11939160a1f5dd3b"}
Oct 30 18:23:39.846284 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.845850    5746 scope.go:122] "RemoveContainer" containerID="59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432"
Oct 30 18:23:39.854126 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.854107    5746 scope.go:122] "RemoveContainer" containerID="59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432"
Oct 30 18:23:39.854400 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:23:39.854382    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432\": container with ID starting with 59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432 not found: ID does not exist" containerID="59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432"
Oct 30 18:23:39.854449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.854410    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432"} err="failed to get container status \"59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432\": rpc error: code = NotFound desc = could not find container \"59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432\": container with ID starting with 59b5ad590ddbe52489c73df52f43189c1a4be9eaa5ced442f8765796fa863432 not found: ID does not exist"
Oct 30 18:23:39.895654 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.895634    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"]
Oct 30 18:23:39.907596 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:39.907573    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6f565b67cf-vsfzv"]
Oct 30 18:23:40.007758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.007731    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef99c56c-d2e7-420c-9d5e-22763166df9f" path="/var/lib/kubelet/pods/ef99c56c-d2e7-420c-9d5e-22763166df9f/volumes"
Oct 30 18:23:40.071274 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.071245    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:23:40.071520 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.071508    5746 cpu_manager.go:414] "containerMap: removing container" podUID="ef99c56c-d2e7-420c-9d5e-22763166df9f" containerName="ta-container"
Oct 30 18:23:40.071559 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.071521    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef99c56c-d2e7-420c-9d5e-22763166df9f" containerName="ta-container"
Oct 30 18:23:40.071593 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.071570    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef99c56c-d2e7-420c-9d5e-22763166df9f" containerName="ta-container"
Oct 30 18:23:40.075853 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.075839    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.090762 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.090743    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:23:40.090882 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.090756    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:23:40.090882 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.090813    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:23:40.090882 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.090825    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:23:40.102440 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.102415    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsdwv\""
Oct 30 18:23:40.102440 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.102415    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-a0f9182a\""
Oct 30 18:23:40.103286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.103264    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:23:40.201543 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.201505    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.201543 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.201545    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.201782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.201569    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qfsr\" (UniqueName: \"kubernetes.io/projected/c133f240-e246-480c-9a86-b120f7c0441f-kube-api-access-9qfsr\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.201782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.201677    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.201782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.201720    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.201782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.201765    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c133f240-e246-480c-9a86-b120f7c0441f-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.302477 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.302440    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.302667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.302497    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.302667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.302529    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c133f240-e246-480c-9a86-b120f7c0441f-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.302667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.302551    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.302667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.302572    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.302667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.302595    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qfsr\" (UniqueName: \"kubernetes.io/projected/c133f240-e246-480c-9a86-b120f7c0441f-kube-api-access-9qfsr\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.303590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.303558    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c133f240-e246-480c-9a86-b120f7c0441f-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.305309 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.305289    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.305425 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.305324    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.305466 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.305432    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.305503 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.305485    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.325921 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.325880    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qfsr\" (UniqueName: \"kubernetes.io/projected/c133f240-e246-480c-9a86-b120f7c0441f-kube-api-access-9qfsr\") pod \"data-science-collector-collector-1\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.387532 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.387462    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:23:40.683465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.683382    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:23:40.686395 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:23:40.686369    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc133f240_e246_480c_9a86_b120f7c0441f.slice/crio-2393de0c577b8850e824bf2f563227ff6f9286c96773acf1bf347663483477d6 WatchSource:0}: Error finding container 2393de0c577b8850e824bf2f563227ff6f9286c96773acf1bf347663483477d6: Status 404 returned error can't find the container with id 2393de0c577b8850e824bf2f563227ff6f9286c96773acf1bf347663483477d6
Oct 30 18:23:40.688559 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.688525    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:23:40.688644 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.688588    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:23:40.852636 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.852593    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"c133f240-e246-480c-9a86-b120f7c0441f","Type":"ContainerStarted","Data":"988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833"}
Oct 30 18:23:40.853189 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.852658    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"c133f240-e246-480c-9a86-b120f7c0441f","Type":"ContainerStarted","Data":"2393de0c577b8850e824bf2f563227ff6f9286c96773acf1bf347663483477d6"}
Oct 30 18:23:40.895546 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:40.895481    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-1" podStartSLOduration=0.895462831 podStartE2EDuration="895.462831ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:23:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:23:40.894852338 +0000 UTC m=+1591.061670502" watchObservedRunningTime="2026-10-30 18:23:40.895462831 +0000 UTC m=+1591.062280958"
Oct 30 18:23:42.789515 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:23:42.789477    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:23:44.366276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.366240    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"]
Oct 30 18:23:44.369827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.369805    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.396979 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.396951    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"]
Oct 30 18:23:44.437819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.437780    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n7jr\" (UniqueName: \"kubernetes.io/projected/07e6a92a-2e60-438b-bc97-4116faf086b2-kube-api-access-5n7jr\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.438026 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.437827    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/07e6a92a-2e60-438b-bc97-4116faf086b2-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.438026 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.437976    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/07e6a92a-2e60-438b-bc97-4116faf086b2-ta-internal\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.538376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.538340    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/07e6a92a-2e60-438b-bc97-4116faf086b2-ta-internal\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.538577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.538401    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n7jr\" (UniqueName: \"kubernetes.io/projected/07e6a92a-2e60-438b-bc97-4116faf086b2-kube-api-access-5n7jr\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.538577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.538429    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/07e6a92a-2e60-438b-bc97-4116faf086b2-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.539099 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.539072    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/07e6a92a-2e60-438b-bc97-4116faf086b2-ta-internal\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.541153 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.541129    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/07e6a92a-2e60-438b-bc97-4116faf086b2-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.561610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.561584    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n7jr\" (UniqueName: \"kubernetes.io/projected/07e6a92a-2e60-438b-bc97-4116faf086b2-kube-api-access-5n7jr\") pod \"data-science-collector-targetallocator-7995b6f9c5-x97tm\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.681654 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.681562    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:44.979757 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.979683    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"]
Oct 30 18:23:44.983141 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:23:44.983115    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07e6a92a_2e60_438b_bc97_4116faf086b2.slice/crio-ae55333e42771485dce496c255378e86c3348e0ec0c095885ddb5373c054f968 WatchSource:0}: Error finding container ae55333e42771485dce496c255378e86c3348e0ec0c095885ddb5373c054f968: Status 404 returned error can't find the container with id ae55333e42771485dce496c255378e86c3348e0ec0c095885ddb5373c054f968
Oct 30 18:23:44.985418 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.985383    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:23:44.985505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:44.985442    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:23:45.874880 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:45.874842    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm" event={"ID":"07e6a92a-2e60-438b-bc97-4116faf086b2","Type":"ContainerStarted","Data":"5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787"}
Oct 30 18:23:45.875413 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:45.874892    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm" event={"ID":"07e6a92a-2e60-438b-bc97-4116faf086b2","Type":"ContainerStarted","Data":"ae55333e42771485dce496c255378e86c3348e0ec0c095885ddb5373c054f968"}
Oct 30 18:23:45.875413 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:45.875291    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:45.876764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:45.876746    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:23:45.917682 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:45.917636    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm" podStartSLOduration=1.9176249950000002 podStartE2EDuration="1.917624995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:23:45.917549212 +0000 UTC m=+1596.084367350" watchObservedRunningTime="2026-10-30 18:23:45.917624995 +0000 UTC m=+1596.084443116"
Oct 30 18:23:45.990083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:45.990051    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"]
Oct 30 18:23:45.991066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:45.990259    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz" podUID="553b242b-afca-46f1-8793-3df008282823" containerName="ta-container" containerID="cri-o://28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075" gracePeriod=30
Oct 30 18:23:46.224179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.224159    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:46.354986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.354900    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/553b242b-afca-46f1-8793-3df008282823-ta-internal\" (UniqueName: \"kubernetes.io/configmap/553b242b-afca-46f1-8793-3df008282823-ta-internal\") pod \"553b242b-afca-46f1-8793-3df008282823\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") "
Oct 30 18:23:46.354986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.354997    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/553b242b-afca-46f1-8793-3df008282823-kube-api-access-9gcz6\" (UniqueName: \"kubernetes.io/projected/553b242b-afca-46f1-8793-3df008282823-kube-api-access-9gcz6\") pod \"553b242b-afca-46f1-8793-3df008282823\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") "
Oct 30 18:23:46.355210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.355156    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/553b242b-afca-46f1-8793-3df008282823-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/553b242b-afca-46f1-8793-3df008282823-data-science-collector-ta-server-cert\") pod \"553b242b-afca-46f1-8793-3df008282823\" (UID: \"553b242b-afca-46f1-8793-3df008282823\") "
Oct 30 18:23:46.355314 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.355292    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/553b242b-afca-46f1-8793-3df008282823-ta-internal" pod "553b242b-afca-46f1-8793-3df008282823" (UID: "553b242b-afca-46f1-8793-3df008282823"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:23:46.355422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.355409    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/553b242b-afca-46f1-8793-3df008282823-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:46.357242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.357215    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/553b242b-afca-46f1-8793-3df008282823-data-science-collector-ta-server-cert" pod "553b242b-afca-46f1-8793-3df008282823" (UID: "553b242b-afca-46f1-8793-3df008282823"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:23:46.357364 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.357243    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/553b242b-afca-46f1-8793-3df008282823-kube-api-access-9gcz6" pod "553b242b-afca-46f1-8793-3df008282823" (UID: "553b242b-afca-46f1-8793-3df008282823"). InnerVolumeSpecName "kube-api-access-9gcz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:23:46.456310 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.456236    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gcz6\" (UniqueName: \"kubernetes.io/projected/553b242b-afca-46f1-8793-3df008282823-kube-api-access-9gcz6\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:46.456310 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.456261    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/553b242b-afca-46f1-8793-3df008282823-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:23:46.880411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.880377    5746 generic.go:356] "Generic (PLEG): container finished" podID="553b242b-afca-46f1-8793-3df008282823" containerID="28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075" exitCode=0
Oct 30 18:23:46.880848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.880426    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"
Oct 30 18:23:46.880848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.880473    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz" event={"ID":"553b242b-afca-46f1-8793-3df008282823","Type":"ContainerDied","Data":"28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075"}
Oct 30 18:23:46.880848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.880513    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz" event={"ID":"553b242b-afca-46f1-8793-3df008282823","Type":"ContainerDied","Data":"69ae2b6aaee5a05311446ff89d003281c0b699a397dd1300d8695d922618b3f7"}
Oct 30 18:23:46.880848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.880541    5746 scope.go:122] "RemoveContainer" containerID="28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075"
Oct 30 18:23:46.888981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.888965    5746 scope.go:122] "RemoveContainer" containerID="28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075"
Oct 30 18:23:46.889229 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:23:46.889211    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075\": container with ID starting with 28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075 not found: ID does not exist" containerID="28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075"
Oct 30 18:23:46.889319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.889239    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075"} err="failed to get container status \"28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075\": rpc error: code = NotFound desc = could not find container \"28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075\": container with ID starting with 28388987fb12d8a088bd470ca99647751325c3957846805ca8b063ecd6af5075 not found: ID does not exist"
Oct 30 18:23:46.930799 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.930777    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"]
Oct 30 18:23:46.943004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:46.942982    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-cd46d58f7-88smz"]
Oct 30 18:23:48.008624 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:23:48.008580    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="553b242b-afca-46f1-8793-3df008282823" path="/var/lib/kubelet/pods/553b242b-afca-46f1-8793-3df008282823/volumes"
Oct 30 18:24:10.276196 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:24:10.276167    5746 scope.go:122] "RemoveContainer" containerID="ad664bdf53ee4746f659880094b166a39a2ee86c4433b7142372cae23b979cee"
Oct 30 18:24:12.789889 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:24:12.789852    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:24:42.790103 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:24:42.790016    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:25:09.159256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.159223    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"]
Oct 30 18:25:09.159657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.159507    5746 cpu_manager.go:414] "containerMap: removing container" podUID="553b242b-afca-46f1-8793-3df008282823" containerName="ta-container"
Oct 30 18:25:09.159657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.159518    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="553b242b-afca-46f1-8793-3df008282823" containerName="ta-container"
Oct 30 18:25:09.159657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.159576    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="553b242b-afca-46f1-8793-3df008282823" containerName="ta-container"
Oct 30 18:25:09.162485 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.162469    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.189989 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.189968    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"]
Oct 30 18:25:09.305385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.305353    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/269cabd7-25de-476f-bb2a-b173a280342e-ta-internal\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.305575 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.305414    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qzm4\" (UniqueName: \"kubernetes.io/projected/269cabd7-25de-476f-bb2a-b173a280342e-kube-api-access-6qzm4\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.305575 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.305469    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/269cabd7-25de-476f-bb2a-b173a280342e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.406225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.406190    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/269cabd7-25de-476f-bb2a-b173a280342e-ta-internal\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.406434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.406248    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qzm4\" (UniqueName: \"kubernetes.io/projected/269cabd7-25de-476f-bb2a-b173a280342e-kube-api-access-6qzm4\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.406434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.406268    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/269cabd7-25de-476f-bb2a-b173a280342e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.406969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.406948    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/269cabd7-25de-476f-bb2a-b173a280342e-ta-internal\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.408880 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.408851    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/269cabd7-25de-476f-bb2a-b173a280342e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.430735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.430671    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qzm4\" (UniqueName: \"kubernetes.io/projected/269cabd7-25de-476f-bb2a-b173a280342e-kube-api-access-6qzm4\") pod \"data-science-collector-targetallocator-bf9b758d-vlr2s\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.474093 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.474060    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:09.771367 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.771287    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"]
Oct 30 18:25:09.774166 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:25:09.774128    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod269cabd7_25de_476f_bb2a_b173a280342e.slice/crio-a60fced65d9798affc7ec6c0437a6bdb12b485bd6d43324c32e72e69c1fe842e WatchSource:0}: Error finding container a60fced65d9798affc7ec6c0437a6bdb12b485bd6d43324c32e72e69c1fe842e: Status 404 returned error can't find the container with id a60fced65d9798affc7ec6c0437a6bdb12b485bd6d43324c32e72e69c1fe842e
Oct 30 18:25:09.776213 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.776183    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:25:09.776335 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:09.776249    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:25:10.202510 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.202478    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s" event={"ID":"269cabd7-25de-476f-bb2a-b173a280342e","Type":"ContainerStarted","Data":"47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383"}
Oct 30 18:25:10.202980 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.202518    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s" event={"ID":"269cabd7-25de-476f-bb2a-b173a280342e","Type":"ContainerStarted","Data":"a60fced65d9798affc7ec6c0437a6bdb12b485bd6d43324c32e72e69c1fe842e"}
Oct 30 18:25:10.202980 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.202636    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:10.204396 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.204372    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:10.286216 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.286167    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s" podStartSLOduration=1.2861530939999999 podStartE2EDuration="1.286153094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:25:10.244537657 +0000 UTC m=+1680.411355784" watchObservedRunningTime="2026-10-30 18:25:10.286153094 +0000 UTC m=+1680.452971220"
Oct 30 18:25:10.314999 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.314968    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"]
Oct 30 18:25:10.315245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.315215    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm" podUID="07e6a92a-2e60-438b-bc97-4116faf086b2" containerName="ta-container" containerID="cri-o://5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787" gracePeriod=30
Oct 30 18:25:10.562536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.562514    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:25:10.719395 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.719363    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07e6a92a-2e60-438b-bc97-4116faf086b2-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/07e6a92a-2e60-438b-bc97-4116faf086b2-data-science-collector-ta-server-cert\") pod \"07e6a92a-2e60-438b-bc97-4116faf086b2\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") "
Oct 30 18:25:10.719578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.719413    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07e6a92a-2e60-438b-bc97-4116faf086b2-kube-api-access-5n7jr\" (UniqueName: \"kubernetes.io/projected/07e6a92a-2e60-438b-bc97-4116faf086b2-kube-api-access-5n7jr\") pod \"07e6a92a-2e60-438b-bc97-4116faf086b2\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") "
Oct 30 18:25:10.719578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.719446    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07e6a92a-2e60-438b-bc97-4116faf086b2-ta-internal\" (UniqueName: \"kubernetes.io/configmap/07e6a92a-2e60-438b-bc97-4116faf086b2-ta-internal\") pod \"07e6a92a-2e60-438b-bc97-4116faf086b2\" (UID: \"07e6a92a-2e60-438b-bc97-4116faf086b2\") "
Oct 30 18:25:10.719855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.719831    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07e6a92a-2e60-438b-bc97-4116faf086b2-ta-internal" pod "07e6a92a-2e60-438b-bc97-4116faf086b2" (UID: "07e6a92a-2e60-438b-bc97-4116faf086b2"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:25:10.721678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.721652    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07e6a92a-2e60-438b-bc97-4116faf086b2-data-science-collector-ta-server-cert" pod "07e6a92a-2e60-438b-bc97-4116faf086b2" (UID: "07e6a92a-2e60-438b-bc97-4116faf086b2"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:25:10.721778 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.721702    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07e6a92a-2e60-438b-bc97-4116faf086b2-kube-api-access-5n7jr" pod "07e6a92a-2e60-438b-bc97-4116faf086b2" (UID: "07e6a92a-2e60-438b-bc97-4116faf086b2"). InnerVolumeSpecName "kube-api-access-5n7jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:25:10.820392 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.820315    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/07e6a92a-2e60-438b-bc97-4116faf086b2-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:10.820392 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.820348    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5n7jr\" (UniqueName: \"kubernetes.io/projected/07e6a92a-2e60-438b-bc97-4116faf086b2-kube-api-access-5n7jr\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:10.820392 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:10.820360    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/07e6a92a-2e60-438b-bc97-4116faf086b2-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:11.207478 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.207446    5746 generic.go:356] "Generic (PLEG): container finished" podID="07e6a92a-2e60-438b-bc97-4116faf086b2" containerID="5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787" exitCode=0
Oct 30 18:25:11.207936 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.207489    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"
Oct 30 18:25:11.207936 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.207531    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm" event={"ID":"07e6a92a-2e60-438b-bc97-4116faf086b2","Type":"ContainerDied","Data":"5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787"}
Oct 30 18:25:11.207936 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.207573    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm" event={"ID":"07e6a92a-2e60-438b-bc97-4116faf086b2","Type":"ContainerDied","Data":"ae55333e42771485dce496c255378e86c3348e0ec0c095885ddb5373c054f968"}
Oct 30 18:25:11.207936 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.207591    5746 scope.go:122] "RemoveContainer" containerID="5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787"
Oct 30 18:25:11.216080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.216062    5746 scope.go:122] "RemoveContainer" containerID="5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787"
Oct 30 18:25:11.216334 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:25:11.216318    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787\": container with ID starting with 5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787 not found: ID does not exist" containerID="5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787"
Oct 30 18:25:11.216380 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.216342    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787"} err="failed to get container status \"5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787\": rpc error: code = NotFound desc = could not find container \"5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787\": container with ID starting with 5f7a0b3218e20b0c041ed9e4ea4ebadf36eac0e6cabd73130aac979d34790787 not found: ID does not exist"
Oct 30 18:25:11.257612 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.257588    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"]
Oct 30 18:25:11.269613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:11.269593    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7995b6f9c5-x97tm"]
Oct 30 18:25:12.008259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:12.008226    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07e6a92a-2e60-438b-bc97-4116faf086b2" path="/var/lib/kubelet/pods/07e6a92a-2e60-438b-bc97-4116faf086b2/volumes"
Oct 30 18:25:12.790311 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:25:12.790277    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:25:15.874192 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:15.874154    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:25:15.874717 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:15.874378    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-1" podUID="c133f240-e246-480c-9a86-b120f7c0441f" containerName="otc-container" containerID="cri-o://988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833" gracePeriod=30
Oct 30 18:25:15.918296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:15.918266    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"]
Oct 30 18:25:15.918559 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:15.918547    5746 cpu_manager.go:414] "containerMap: removing container" podUID="07e6a92a-2e60-438b-bc97-4116faf086b2" containerName="ta-container"
Oct 30 18:25:15.918611 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:15.918560    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07e6a92a-2e60-438b-bc97-4116faf086b2" containerName="ta-container"
Oct 30 18:25:15.918651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:15.918623    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="07e6a92a-2e60-438b-bc97-4116faf086b2" containerName="ta-container"
Oct 30 18:25:15.923197 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:15.923174    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:15.942149 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:15.942121    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"]
Oct 30 18:25:16.064172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.064141    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.064337 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.064182    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-ta-internal\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.064337 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.064257    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8mhh\" (UniqueName: \"kubernetes.io/projected/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-kube-api-access-f8mhh\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.126074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.125997    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"]
Oct 30 18:25:16.126211 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:25:16.126191    5746 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[data-science-collector-ta-server-cert kube-api-access-f8mhh ta-internal], unattached volumes=[], failed to process volumes=[]: context canceled" pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84" podUID="f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e"
Oct 30 18:25:16.165204 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.165175    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.165340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.165212    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-ta-internal\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.165340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.165253    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8mhh\" (UniqueName: \"kubernetes.io/projected/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-kube-api-access-f8mhh\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.165944 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.165900    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-ta-internal\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.167683 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.167662    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.173217 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.173194    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"]
Oct 30 18:25:16.176579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.176564    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.228702 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.228675    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.234488 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.234464    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"]
Oct 30 18:25:16.234798 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.234777    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.240924 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.240883    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8mhh\" (UniqueName: \"kubernetes.io/projected/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-kube-api-access-f8mhh\") pod \"data-science-collector-targetallocator-547cd8c557-6gr84\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:16.265943 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.265896    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbm2n\" (UniqueName: \"kubernetes.io/projected/08f31d7f-1716-4579-896c-451679908fa3-kube-api-access-gbm2n\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.266089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.265959    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/08f31d7f-1716-4579-896c-451679908fa3-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.266134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.266108    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/08f31d7f-1716-4579-896c-451679908fa3-ta-internal\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.366750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.366715    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-ta-internal\" (UniqueName: \"kubernetes.io/configmap/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-ta-internal\") pod \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") "
Oct 30 18:25:16.366969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.366769    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-data-science-collector-ta-server-cert\") pod \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") "
Oct 30 18:25:16.366969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.366855    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-kube-api-access-f8mhh\" (UniqueName: \"kubernetes.io/projected/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-kube-api-access-f8mhh\") pod \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\" (UID: \"f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e\") "
Oct 30 18:25:16.367115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.367000    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/08f31d7f-1716-4579-896c-451679908fa3-ta-internal\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.367115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.367041    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbm2n\" (UniqueName: \"kubernetes.io/projected/08f31d7f-1716-4579-896c-451679908fa3-kube-api-access-gbm2n\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.367115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.367087    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/08f31d7f-1716-4579-896c-451679908fa3-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.367265 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.367175    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-ta-internal" pod "f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e" (UID: "f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:25:16.367746 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.367714    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/08f31d7f-1716-4579-896c-451679908fa3-ta-internal\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.369590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.369566    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-data-science-collector-ta-server-cert" pod "f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e" (UID: "f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:25:16.369590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.369579    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-kube-api-access-f8mhh" pod "f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e" (UID: "f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e"). InnerVolumeSpecName "kube-api-access-f8mhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:25:16.369770 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.369748    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/08f31d7f-1716-4579-896c-451679908fa3-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.390519 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.390461    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbm2n\" (UniqueName: \"kubernetes.io/projected/08f31d7f-1716-4579-896c-451679908fa3-kube-api-access-gbm2n\") pod \"data-science-collector-targetallocator-c654b8cb4-hxf6f\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.467677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.467642    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:16.467677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.467670    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:16.467677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.467684    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8mhh\" (UniqueName: \"kubernetes.io/projected/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e-kube-api-access-f8mhh\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:16.487608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.487580    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:16.781974 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.781885    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"]
Oct 30 18:25:16.785032 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:25:16.785005    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08f31d7f_1716_4579_896c_451679908fa3.slice/crio-9fcfb70f877109665a60f482b357ec6b1c666ad5ae0586dd7a82c629e79d4ef0 WatchSource:0}: Error finding container 9fcfb70f877109665a60f482b357ec6b1c666ad5ae0586dd7a82c629e79d4ef0: Status 404 returned error can't find the container with id 9fcfb70f877109665a60f482b357ec6b1c666ad5ae0586dd7a82c629e79d4ef0
Oct 30 18:25:16.787106 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.787075    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:25:16.787202 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:16.787132    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:25:17.233631 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.233602    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"
Oct 30 18:25:17.234100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.233633    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f" event={"ID":"08f31d7f-1716-4579-896c-451679908fa3","Type":"ContainerStarted","Data":"5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8"}
Oct 30 18:25:17.234100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.233684    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f" event={"ID":"08f31d7f-1716-4579-896c-451679908fa3","Type":"ContainerStarted","Data":"9fcfb70f877109665a60f482b357ec6b1c666ad5ae0586dd7a82c629e79d4ef0"}
Oct 30 18:25:17.234100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.233842    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:17.235449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.235426    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:25:17.290600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.290551    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f" podStartSLOduration=1.290538402 podStartE2EDuration="1.290538402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:25:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:25:17.289993497 +0000 UTC m=+1687.456811643" watchObservedRunningTime="2026-10-30 18:25:17.290538402 +0000 UTC m=+1687.457356523"
Oct 30 18:25:17.363955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.363899    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"]
Oct 30 18:25:17.364141 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.364085    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s" podUID="269cabd7-25de-476f-bb2a-b173a280342e" containerName="ta-container" containerID="cri-o://47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383" gracePeriod=30
Oct 30 18:25:17.395587 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.395550    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"]
Oct 30 18:25:17.403708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:17.403676    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-547cd8c557-6gr84"]
Oct 30 18:25:18.008450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.008419    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e" path="/var/lib/kubelet/pods/f464ec9b-c878-4bf3-a2ca-d5ed6f53e57e/volumes"
Oct 30 18:25:18.217823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.217796    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:25:18.245615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.245582    5746 generic.go:356] "Generic (PLEG): container finished" podID="c133f240-e246-480c-9a86-b120f7c0441f" containerID="988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833" exitCode=0
Oct 30 18:25:18.246020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.245645    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:25:18.246020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.245659    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"c133f240-e246-480c-9a86-b120f7c0441f","Type":"ContainerDied","Data":"988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833"}
Oct 30 18:25:18.246020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.245746    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"c133f240-e246-480c-9a86-b120f7c0441f","Type":"ContainerDied","Data":"2393de0c577b8850e824bf2f563227ff6f9286c96773acf1bf347663483477d6"}
Oct 30 18:25:18.246020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.245770    5746 scope.go:122] "RemoveContainer" containerID="988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833"
Oct 30 18:25:18.253827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.253802    5746 scope.go:122] "RemoveContainer" containerID="988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833"
Oct 30 18:25:18.254065 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:25:18.254038    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833\": container with ID starting with 988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833 not found: ID does not exist" containerID="988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833"
Oct 30 18:25:18.254129 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.254073    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833"} err="failed to get container status \"988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833\": rpc error: code = NotFound desc = could not find container \"988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833\": container with ID starting with 988bfdff3ffc8d168df71d8105d96b2473dc558ada8ca33dbe4e4d355b65b833 not found: ID does not exist"
Oct 30 18:25:18.384166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.384139    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-otlp-tls-certs\") pod \"c133f240-e246-480c-9a86-b120f7c0441f\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") "
Oct 30 18:25:18.384350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.384195    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-tls-certs\") pod \"c133f240-e246-480c-9a86-b120f7c0441f\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") "
Oct 30 18:25:18.384350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.384226    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-monitoring-tls-certs\") pod \"c133f240-e246-480c-9a86-b120f7c0441f\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") "
Oct 30 18:25:18.384350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.384257    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c133f240-e246-480c-9a86-b120f7c0441f-otc-internal\" (UniqueName: \"kubernetes.io/configmap/c133f240-e246-480c-9a86-b120f7c0441f-otc-internal\") pod \"c133f240-e246-480c-9a86-b120f7c0441f\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") "
Oct 30 18:25:18.384350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.384316    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-data-science-collector-ta-client-cert\") pod \"c133f240-e246-480c-9a86-b120f7c0441f\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") "
Oct 30 18:25:18.384571 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.384396    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c133f240-e246-480c-9a86-b120f7c0441f-kube-api-access-9qfsr\" (UniqueName: \"kubernetes.io/projected/c133f240-e246-480c-9a86-b120f7c0441f-kube-api-access-9qfsr\") pod \"c133f240-e246-480c-9a86-b120f7c0441f\" (UID: \"c133f240-e246-480c-9a86-b120f7c0441f\") "
Oct 30 18:25:18.384739 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.384710    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c133f240-e246-480c-9a86-b120f7c0441f-otc-internal" pod "c133f240-e246-480c-9a86-b120f7c0441f" (UID: "c133f240-e246-480c-9a86-b120f7c0441f"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:25:18.386744 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.386719    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-otlp-tls-certs" pod "c133f240-e246-480c-9a86-b120f7c0441f" (UID: "c133f240-e246-480c-9a86-b120f7c0441f"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:25:18.386862 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.386740    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c133f240-e246-480c-9a86-b120f7c0441f-kube-api-access-9qfsr" pod "c133f240-e246-480c-9a86-b120f7c0441f" (UID: "c133f240-e246-480c-9a86-b120f7c0441f"). InnerVolumeSpecName "kube-api-access-9qfsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:25:18.386862 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.386762    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-data-science-collector-ta-client-cert" pod "c133f240-e246-480c-9a86-b120f7c0441f" (UID: "c133f240-e246-480c-9a86-b120f7c0441f"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:25:18.387075 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.387059    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-tls-certs" pod "c133f240-e246-480c-9a86-b120f7c0441f" (UID: "c133f240-e246-480c-9a86-b120f7c0441f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:25:18.387124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.387104    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-monitoring-tls-certs" pod "c133f240-e246-480c-9a86-b120f7c0441f" (UID: "c133f240-e246-480c-9a86-b120f7c0441f"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:25:18.485100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.485063    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qfsr\" (UniqueName: \"kubernetes.io/projected/c133f240-e246-480c-9a86-b120f7c0441f-kube-api-access-9qfsr\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:18.485100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.485095    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:18.485100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.485106    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:18.485335 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.485115    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:18.485335 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.485124    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/c133f240-e246-480c-9a86-b120f7c0441f-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:18.485335 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.485135    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/c133f240-e246-480c-9a86-b120f7c0441f-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:18.588143 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.588111    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:25:18.600144 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:18.600114    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:25:20.008513 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.008482    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c133f240-e246-480c-9a86-b120f7c0441f" path="/var/lib/kubelet/pods/c133f240-e246-480c-9a86-b120f7c0441f/volumes"
Oct 30 18:25:20.208595 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.208572    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:20.256538 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.256503    5746 generic.go:356] "Generic (PLEG): container finished" podID="269cabd7-25de-476f-bb2a-b173a280342e" containerID="47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383" exitCode=0
Oct 30 18:25:20.256681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.256546    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"
Oct 30 18:25:20.256681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.256550    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s" event={"ID":"269cabd7-25de-476f-bb2a-b173a280342e","Type":"ContainerDied","Data":"47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383"}
Oct 30 18:25:20.256681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.256588    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s" event={"ID":"269cabd7-25de-476f-bb2a-b173a280342e","Type":"ContainerDied","Data":"a60fced65d9798affc7ec6c0437a6bdb12b485bd6d43324c32e72e69c1fe842e"}
Oct 30 18:25:20.256681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.256610    5746 scope.go:122] "RemoveContainer" containerID="47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383"
Oct 30 18:25:20.265863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.265840    5746 scope.go:122] "RemoveContainer" containerID="47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383"
Oct 30 18:25:20.266153 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:25:20.266133    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383\": container with ID starting with 47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383 not found: ID does not exist" containerID="47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383"
Oct 30 18:25:20.266217 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.266164    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383"} err="failed to get container status \"47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383\": rpc error: code = NotFound desc = could not find container \"47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383\": container with ID starting with 47c0568e89fc8c4fbe8d28cfe2245dfee5bbec03c016ce26cebdf117745e1383 not found: ID does not exist"
Oct 30 18:25:20.301824 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.301793    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/269cabd7-25de-476f-bb2a-b173a280342e-ta-internal\" (UniqueName: \"kubernetes.io/configmap/269cabd7-25de-476f-bb2a-b173a280342e-ta-internal\") pod \"269cabd7-25de-476f-bb2a-b173a280342e\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") "
Oct 30 18:25:20.301993 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.301861    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/269cabd7-25de-476f-bb2a-b173a280342e-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/269cabd7-25de-476f-bb2a-b173a280342e-data-science-collector-ta-server-cert\") pod \"269cabd7-25de-476f-bb2a-b173a280342e\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") "
Oct 30 18:25:20.301993 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.301953    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/269cabd7-25de-476f-bb2a-b173a280342e-kube-api-access-6qzm4\" (UniqueName: \"kubernetes.io/projected/269cabd7-25de-476f-bb2a-b173a280342e-kube-api-access-6qzm4\") pod \"269cabd7-25de-476f-bb2a-b173a280342e\" (UID: \"269cabd7-25de-476f-bb2a-b173a280342e\") "
Oct 30 18:25:20.302218 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.302191    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/269cabd7-25de-476f-bb2a-b173a280342e-ta-internal" pod "269cabd7-25de-476f-bb2a-b173a280342e" (UID: "269cabd7-25de-476f-bb2a-b173a280342e"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:25:20.304108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.304078    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/269cabd7-25de-476f-bb2a-b173a280342e-kube-api-access-6qzm4" pod "269cabd7-25de-476f-bb2a-b173a280342e" (UID: "269cabd7-25de-476f-bb2a-b173a280342e"). InnerVolumeSpecName "kube-api-access-6qzm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:25:20.304214 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.304136    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/269cabd7-25de-476f-bb2a-b173a280342e-data-science-collector-ta-server-cert" pod "269cabd7-25de-476f-bb2a-b173a280342e" (UID: "269cabd7-25de-476f-bb2a-b173a280342e"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:25:20.402635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.402609    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qzm4\" (UniqueName: \"kubernetes.io/projected/269cabd7-25de-476f-bb2a-b173a280342e-kube-api-access-6qzm4\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:20.402635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.402634    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/269cabd7-25de-476f-bb2a-b173a280342e-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:20.402812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.402643    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/269cabd7-25de-476f-bb2a-b173a280342e-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:25:20.599922 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.599884    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"]
Oct 30 18:25:20.604374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:20.604345    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s"]
Oct 30 18:25:21.203951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:21.203884    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-bf9b758d-vlr2s" podUID="269cabd7-25de-476f-bb2a-b173a280342e" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.54:8080/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:25:22.008705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.008669    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="269cabd7-25de-476f-bb2a-b173a280342e" path="/var/lib/kubelet/pods/269cabd7-25de-476f-bb2a-b173a280342e/volumes"
Oct 30 18:25:22.312032 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.311956    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:25:22.312421 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.312238    5746 cpu_manager.go:414] "containerMap: removing container" podUID="269cabd7-25de-476f-bb2a-b173a280342e" containerName="ta-container"
Oct 30 18:25:22.312421 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.312248    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="269cabd7-25de-476f-bb2a-b173a280342e" containerName="ta-container"
Oct 30 18:25:22.312421 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.312257    5746 cpu_manager.go:414] "containerMap: removing container" podUID="c133f240-e246-480c-9a86-b120f7c0441f" containerName="otc-container"
Oct 30 18:25:22.312421 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.312263    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c133f240-e246-480c-9a86-b120f7c0441f" containerName="otc-container"
Oct 30 18:25:22.312421 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.312309    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="269cabd7-25de-476f-bb2a-b173a280342e" containerName="ta-container"
Oct 30 18:25:22.312421 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.312317    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="c133f240-e246-480c-9a86-b120f7c0441f" containerName="otc-container"
Oct 30 18:25:22.316639 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.316620    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.343849 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.343821    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:25:22.344062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.343853    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:25:22.344062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.343877    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:25:22.344062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.344020    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-db539ca7\""
Oct 30 18:25:22.344239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.344126    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:25:22.344239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.344184    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsdwv\""
Oct 30 18:25:22.344719 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.344701    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:25:22.418001 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.417967    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.418157 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.418011    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.418157 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.418029    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.418234 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.418186    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/72f2cf56-9058-45b2-8c66-b604d25e388e-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.418234 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.418223    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbtdv\" (UniqueName: \"kubernetes.io/projected/72f2cf56-9058-45b2-8c66-b604d25e388e-kube-api-access-cbtdv\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.418291 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.418257    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.519092 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.519053    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.519269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.519106    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.519269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.519132    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.519269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.519197    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/72f2cf56-9058-45b2-8c66-b604d25e388e-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.519269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.519232    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbtdv\" (UniqueName: \"kubernetes.io/projected/72f2cf56-9058-45b2-8c66-b604d25e388e-kube-api-access-cbtdv\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.519269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.519259    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.520070 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.519980    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/72f2cf56-9058-45b2-8c66-b604d25e388e-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.521833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.521807    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.522015 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.521997    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.522263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.522248    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.522320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.522303    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.542725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.542700    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbtdv\" (UniqueName: \"kubernetes.io/projected/72f2cf56-9058-45b2-8c66-b604d25e388e-kube-api-access-cbtdv\") pod \"data-science-collector-collector-0\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.628346 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.628319    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:25:22.928373 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.928301    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:25:22.931462 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:25:22.931441    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f2cf56_9058_45b2_8c66_b604d25e388e.slice/crio-5c5f383e68e62323266ad86f1e2aae1c298d05e590e623c73a808198f03f229c WatchSource:0}: Error finding container 5c5f383e68e62323266ad86f1e2aae1c298d05e590e623c73a808198f03f229c: Status 404 returned error can't find the container with id 5c5f383e68e62323266ad86f1e2aae1c298d05e590e623c73a808198f03f229c
Oct 30 18:25:22.933500 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.933464    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:25:22.933584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:22.933529    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:25:23.275319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:23.275215    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"72f2cf56-9058-45b2-8c66-b604d25e388e","Type":"ContainerStarted","Data":"a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f"}
Oct 30 18:25:23.275473 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:25:23.275323    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"72f2cf56-9058-45b2-8c66-b604d25e388e","Type":"ContainerStarted","Data":"5c5f383e68e62323266ad86f1e2aae1c298d05e590e623c73a808198f03f229c"}
Oct 30 18:25:42.789877 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:25:42.789839    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:26:12.790011 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:26:12.789979    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:26:42.789549 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:26:42.789510    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:27:12.789160 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:27:12.789123    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:27:27.771321 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.771215    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=125.771194927 podStartE2EDuration="2m5.771194927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:25:23.317525772 +0000 UTC m=+1693.484343899" watchObservedRunningTime="2026-10-30 18:27:27.771194927 +0000 UTC m=+1817.938013054"
Oct 30 18:27:27.772000 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.771980    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"]
Oct 30 18:27:27.775334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.775315    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:27.802446 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.802421    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"]
Oct 30 18:27:27.950128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.950091    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"]
Oct 30 18:27:27.950307 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:27:27.950292    5746 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[data-science-collector-ta-server-cert kube-api-access-cmscj ta-internal], unattached volumes=[], failed to process volumes=[]: context canceled" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2" podUID="f1eced8f-511b-4dcd-b01e-e904e493b8ef"
Oct 30 18:27:27.961576 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.961548    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmscj\" (UniqueName: \"kubernetes.io/projected/f1eced8f-511b-4dcd-b01e-e904e493b8ef-kube-api-access-cmscj\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:27.961708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.961596    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f1eced8f-511b-4dcd-b01e-e904e493b8ef-ta-internal\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:27.961708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.961663    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f1eced8f-511b-4dcd-b01e-e904e493b8ef-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:27.997025 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:27.997002    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"]
Oct 30 18:27:28.000292 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.000276    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.020176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.020149    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"]
Oct 30 18:27:28.063183 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.063100    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f1eced8f-511b-4dcd-b01e-e904e493b8ef-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:28.063183 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.063148    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmscj\" (UniqueName: \"kubernetes.io/projected/f1eced8f-511b-4dcd-b01e-e904e493b8ef-kube-api-access-cmscj\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:28.063422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.063195    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f1eced8f-511b-4dcd-b01e-e904e493b8ef-ta-internal\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:28.063886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.063804    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f1eced8f-511b-4dcd-b01e-e904e493b8ef-ta-internal\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:28.065701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.065678    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f1eced8f-511b-4dcd-b01e-e904e493b8ef-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:28.085920 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.085885    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmscj\" (UniqueName: \"kubernetes.io/projected/f1eced8f-511b-4dcd-b01e-e904e493b8ef-kube-api-access-cmscj\") pod \"data-science-collector-targetallocator-6b75865df-x9sz2\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:28.164134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.164103    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v79vh\" (UniqueName: \"kubernetes.io/projected/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-kube-api-access-v79vh\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.164253 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.164158    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-ta-internal\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.164295 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.164277    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.265443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.265411    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.265594 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.265464    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v79vh\" (UniqueName: \"kubernetes.io/projected/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-kube-api-access-v79vh\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.265594 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.265522    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-ta-internal\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.266231 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.266211    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-ta-internal\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.268107 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.268087    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.288649 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.288626    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v79vh\" (UniqueName: \"kubernetes.io/projected/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-kube-api-access-v79vh\") pod \"data-science-collector-targetallocator-54b49c7998-wxntl\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.311444 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.311418    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.606059 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.606018    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"]
Oct 30 18:27:28.609057 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:27:28.609013    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bc2ec1c_3604_4515_ba9a_f28aafafdc9f.slice/crio-11ec6e378f4915cd0e7794333b1b16ffd35711f7dccf0497762888272428afee WatchSource:0}: Error finding container 11ec6e378f4915cd0e7794333b1b16ffd35711f7dccf0497762888272428afee: Status 404 returned error can't find the container with id 11ec6e378f4915cd0e7794333b1b16ffd35711f7dccf0497762888272428afee
Oct 30 18:27:28.611294 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.611262    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:28.611375 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.611321    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:28.767269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.767237    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:28.767469 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.767263    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl" event={"ID":"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f","Type":"ContainerStarted","Data":"edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c"}
Oct 30 18:27:28.767469 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.767314    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:28.767469 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.767329    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl" event={"ID":"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f","Type":"ContainerStarted","Data":"11ec6e378f4915cd0e7794333b1b16ffd35711f7dccf0497762888272428afee"}
Oct 30 18:27:28.768551 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.768518    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl" podUID="1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.59:8080/readyz\": dial tcp 10.133.0.59:8080: connect: connection refused"
Oct 30 18:27:28.773881 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.773859    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:28.808353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.808303    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl" podStartSLOduration=1.808290864 podStartE2EDuration="1.808290864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:27:28.807686075 +0000 UTC m=+1818.974504241" watchObservedRunningTime="2026-10-30 18:27:28.808290864 +0000 UTC m=+1818.975109043"
Oct 30 18:27:28.869742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.869671    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1eced8f-511b-4dcd-b01e-e904e493b8ef-ta-internal\" (UniqueName: \"kubernetes.io/configmap/f1eced8f-511b-4dcd-b01e-e904e493b8ef-ta-internal\") pod \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") "
Oct 30 18:27:28.869742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.869716    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1eced8f-511b-4dcd-b01e-e904e493b8ef-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f1eced8f-511b-4dcd-b01e-e904e493b8ef-data-science-collector-ta-server-cert\") pod \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") "
Oct 30 18:27:28.869945 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.869764    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1eced8f-511b-4dcd-b01e-e904e493b8ef-kube-api-access-cmscj\" (UniqueName: \"kubernetes.io/projected/f1eced8f-511b-4dcd-b01e-e904e493b8ef-kube-api-access-cmscj\") pod \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\" (UID: \"f1eced8f-511b-4dcd-b01e-e904e493b8ef\") "
Oct 30 18:27:28.870110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.870083    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1eced8f-511b-4dcd-b01e-e904e493b8ef-ta-internal" pod "f1eced8f-511b-4dcd-b01e-e904e493b8ef" (UID: "f1eced8f-511b-4dcd-b01e-e904e493b8ef"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:27:28.872023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.871997    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1eced8f-511b-4dcd-b01e-e904e493b8ef-kube-api-access-cmscj" pod "f1eced8f-511b-4dcd-b01e-e904e493b8ef" (UID: "f1eced8f-511b-4dcd-b01e-e904e493b8ef"). InnerVolumeSpecName "kube-api-access-cmscj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:27:28.872114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.872038    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1eced8f-511b-4dcd-b01e-e904e493b8ef-data-science-collector-ta-server-cert" pod "f1eced8f-511b-4dcd-b01e-e904e493b8ef" (UID: "f1eced8f-511b-4dcd-b01e-e904e493b8ef"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:28.970843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.970814    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f1eced8f-511b-4dcd-b01e-e904e493b8ef-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:28.970843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.970841    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f1eced8f-511b-4dcd-b01e-e904e493b8ef-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:28.971023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:28.970857    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmscj\" (UniqueName: \"kubernetes.io/projected/f1eced8f-511b-4dcd-b01e-e904e493b8ef-kube-api-access-cmscj\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:29.771248 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:29.771218    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"
Oct 30 18:27:29.772544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:29.772520    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:29.848170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:29.848133    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"]
Oct 30 18:27:29.853055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:29.853032    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6b75865df-x9sz2"]
Oct 30 18:27:29.927160 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:29.927132    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"]
Oct 30 18:27:29.927360 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:29.927316    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f" podUID="08f31d7f-1716-4579-896c-451679908fa3" containerName="ta-container" containerID="cri-o://5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8" gracePeriod=30
Oct 30 18:27:30.008304 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.008271    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1eced8f-511b-4dcd-b01e-e904e493b8ef" path="/var/lib/kubelet/pods/f1eced8f-511b-4dcd-b01e-e904e493b8ef/volumes"
Oct 30 18:27:30.177180 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.177160    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:27:30.280668 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.280637    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08f31d7f-1716-4579-896c-451679908fa3-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/08f31d7f-1716-4579-896c-451679908fa3-data-science-collector-ta-server-cert\") pod \"08f31d7f-1716-4579-896c-451679908fa3\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") "
Oct 30 18:27:30.280823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.280705    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08f31d7f-1716-4579-896c-451679908fa3-ta-internal\" (UniqueName: \"kubernetes.io/configmap/08f31d7f-1716-4579-896c-451679908fa3-ta-internal\") pod \"08f31d7f-1716-4579-896c-451679908fa3\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") "
Oct 30 18:27:30.280823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.280744    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08f31d7f-1716-4579-896c-451679908fa3-kube-api-access-gbm2n\" (UniqueName: \"kubernetes.io/projected/08f31d7f-1716-4579-896c-451679908fa3-kube-api-access-gbm2n\") pod \"08f31d7f-1716-4579-896c-451679908fa3\" (UID: \"08f31d7f-1716-4579-896c-451679908fa3\") "
Oct 30 18:27:30.281135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.281092    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f31d7f-1716-4579-896c-451679908fa3-ta-internal" pod "08f31d7f-1716-4579-896c-451679908fa3" (UID: "08f31d7f-1716-4579-896c-451679908fa3"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:27:30.282802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.282777    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08f31d7f-1716-4579-896c-451679908fa3-kube-api-access-gbm2n" pod "08f31d7f-1716-4579-896c-451679908fa3" (UID: "08f31d7f-1716-4579-896c-451679908fa3"). InnerVolumeSpecName "kube-api-access-gbm2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:27:30.282955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.282925    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08f31d7f-1716-4579-896c-451679908fa3-data-science-collector-ta-server-cert" pod "08f31d7f-1716-4579-896c-451679908fa3" (UID: "08f31d7f-1716-4579-896c-451679908fa3"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:30.382137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.382108    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/08f31d7f-1716-4579-896c-451679908fa3-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:30.382137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.382133    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbm2n\" (UniqueName: \"kubernetes.io/projected/08f31d7f-1716-4579-896c-451679908fa3-kube-api-access-gbm2n\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:30.382137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.382143    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/08f31d7f-1716-4579-896c-451679908fa3-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:30.776490 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.776397    5746 generic.go:356] "Generic (PLEG): container finished" podID="08f31d7f-1716-4579-896c-451679908fa3" containerID="5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8" exitCode=0
Oct 30 18:27:30.776490 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.776445    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"
Oct 30 18:27:30.776741 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.776487    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f" event={"ID":"08f31d7f-1716-4579-896c-451679908fa3","Type":"ContainerDied","Data":"5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8"}
Oct 30 18:27:30.776741 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.776525    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f" event={"ID":"08f31d7f-1716-4579-896c-451679908fa3","Type":"ContainerDied","Data":"9fcfb70f877109665a60f482b357ec6b1c666ad5ae0586dd7a82c629e79d4ef0"}
Oct 30 18:27:30.776741 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.776541    5746 scope.go:122] "RemoveContainer" containerID="5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8"
Oct 30 18:27:30.785131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.785115    5746 scope.go:122] "RemoveContainer" containerID="5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8"
Oct 30 18:27:30.785391 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:27:30.785375    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8\": container with ID starting with 5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8 not found: ID does not exist" containerID="5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8"
Oct 30 18:27:30.785453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.785402    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8"} err="failed to get container status \"5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8\": rpc error: code = NotFound desc = could not find container \"5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8\": container with ID starting with 5f46e9e99dddaa869468f142511b9699e9798fdf1d3bde6c620097918c148aa8 not found: ID does not exist"
Oct 30 18:27:30.825701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.825673    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"]
Oct 30 18:27:30.837679 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:30.837654    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-c654b8cb4-hxf6f"]
Oct 30 18:27:31.896561 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:31.896320    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:31.897171 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:31.896595    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="72f2cf56-9058-45b2-8c66-b604d25e388e" containerName="otc-container" containerID="cri-o://a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f" gracePeriod=30
Oct 30 18:27:32.007897 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:32.007862    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08f31d7f-1716-4579-896c-451679908fa3" path="/var/lib/kubelet/pods/08f31d7f-1716-4579-896c-451679908fa3/volumes"
Oct 30 18:27:34.235548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.235523    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.313410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.313337    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-monitoring-tls-certs\") pod \"72f2cf56-9058-45b2-8c66-b604d25e388e\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") "
Oct 30 18:27:34.313410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.313375    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72f2cf56-9058-45b2-8c66-b604d25e388e-otc-internal\" (UniqueName: \"kubernetes.io/configmap/72f2cf56-9058-45b2-8c66-b604d25e388e-otc-internal\") pod \"72f2cf56-9058-45b2-8c66-b604d25e388e\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") "
Oct 30 18:27:34.313410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.313407    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-otlp-tls-certs\") pod \"72f2cf56-9058-45b2-8c66-b604d25e388e\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") "
Oct 30 18:27:34.313677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.313428    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-data-science-collector-ta-client-cert\") pod \"72f2cf56-9058-45b2-8c66-b604d25e388e\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") "
Oct 30 18:27:34.313677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.313622    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72f2cf56-9058-45b2-8c66-b604d25e388e-kube-api-access-cbtdv\" (UniqueName: \"kubernetes.io/projected/72f2cf56-9058-45b2-8c66-b604d25e388e-kube-api-access-cbtdv\") pod \"72f2cf56-9058-45b2-8c66-b604d25e388e\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") "
Oct 30 18:27:34.313804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.313681    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-tls-certs\") pod \"72f2cf56-9058-45b2-8c66-b604d25e388e\" (UID: \"72f2cf56-9058-45b2-8c66-b604d25e388e\") "
Oct 30 18:27:34.313871 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.313829    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72f2cf56-9058-45b2-8c66-b604d25e388e-otc-internal" pod "72f2cf56-9058-45b2-8c66-b604d25e388e" (UID: "72f2cf56-9058-45b2-8c66-b604d25e388e"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:27:34.314050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.314023    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/72f2cf56-9058-45b2-8c66-b604d25e388e-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:34.315812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.315789    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-monitoring-tls-certs" pod "72f2cf56-9058-45b2-8c66-b604d25e388e" (UID: "72f2cf56-9058-45b2-8c66-b604d25e388e"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:34.316290 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.316264    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-data-science-collector-ta-client-cert" pod "72f2cf56-9058-45b2-8c66-b604d25e388e" (UID: "72f2cf56-9058-45b2-8c66-b604d25e388e"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:34.316374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.316271    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f2cf56-9058-45b2-8c66-b604d25e388e-kube-api-access-cbtdv" pod "72f2cf56-9058-45b2-8c66-b604d25e388e" (UID: "72f2cf56-9058-45b2-8c66-b604d25e388e"). InnerVolumeSpecName "kube-api-access-cbtdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:27:34.316374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.316317    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-otlp-tls-certs" pod "72f2cf56-9058-45b2-8c66-b604d25e388e" (UID: "72f2cf56-9058-45b2-8c66-b604d25e388e"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:34.316374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.316340    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-tls-certs" pod "72f2cf56-9058-45b2-8c66-b604d25e388e" (UID: "72f2cf56-9058-45b2-8c66-b604d25e388e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:34.415065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.415034    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:34.415065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.415061    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:34.415249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.415073    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:34.415249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.415084    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbtdv\" (UniqueName: \"kubernetes.io/projected/72f2cf56-9058-45b2-8c66-b604d25e388e-kube-api-access-cbtdv\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:34.415249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.415095    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f2cf56-9058-45b2-8c66-b604d25e388e-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:34.800128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.800099    5746 generic.go:356] "Generic (PLEG): container finished" podID="72f2cf56-9058-45b2-8c66-b604d25e388e" containerID="a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f" exitCode=0
Oct 30 18:27:34.800288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.800153    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.800288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.800169    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"72f2cf56-9058-45b2-8c66-b604d25e388e","Type":"ContainerDied","Data":"a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f"}
Oct 30 18:27:34.800288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.800212    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"72f2cf56-9058-45b2-8c66-b604d25e388e","Type":"ContainerDied","Data":"5c5f383e68e62323266ad86f1e2aae1c298d05e590e623c73a808198f03f229c"}
Oct 30 18:27:34.800288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.800233    5746 scope.go:122] "RemoveContainer" containerID="a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f"
Oct 30 18:27:34.808798 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.808776    5746 scope.go:122] "RemoveContainer" containerID="a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f"
Oct 30 18:27:34.809073 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:27:34.809054    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f\": container with ID starting with a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f not found: ID does not exist" containerID="a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f"
Oct 30 18:27:34.809147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.809087    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f"} err="failed to get container status \"a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f\": rpc error: code = NotFound desc = could not find container \"a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f\": container with ID starting with a98918fc9c07d773c2ac6bd1b3e95bf1c2ce24980877d3082cf24defb3fbe20f not found: ID does not exist"
Oct 30 18:27:34.850139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.850114    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:34.862663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.862641    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:34.896864 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.896836    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:34.897136 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.897123    5746 cpu_manager.go:414] "containerMap: removing container" podUID="08f31d7f-1716-4579-896c-451679908fa3" containerName="ta-container"
Oct 30 18:27:34.897191 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.897137    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08f31d7f-1716-4579-896c-451679908fa3" containerName="ta-container"
Oct 30 18:27:34.897191 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.897148    5746 cpu_manager.go:414] "containerMap: removing container" podUID="72f2cf56-9058-45b2-8c66-b604d25e388e" containerName="otc-container"
Oct 30 18:27:34.897191 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.897153    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72f2cf56-9058-45b2-8c66-b604d25e388e" containerName="otc-container"
Oct 30 18:27:34.897326 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.897227    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="72f2cf56-9058-45b2-8c66-b604d25e388e" containerName="otc-container"
Oct 30 18:27:34.897326 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.897251    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="08f31d7f-1716-4579-896c-451679908fa3" containerName="ta-container"
Oct 30 18:27:34.901651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.901635    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.917179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.917160    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:27:34.917379 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.917361    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:27:34.917449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.917412    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsdwv\""
Oct 30 18:27:34.917449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.917422    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:27:34.917449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.917423    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:27:34.917575 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.917446    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-a0f9182a\""
Oct 30 18:27:34.920071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.920023    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.920194 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.920073    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.920194 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.920117    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.920306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.920209    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.920306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.920244    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.920306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.920272    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8qfc\" (UniqueName: \"kubernetes.io/projected/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-kube-api-access-q8qfc\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:34.921123 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:34.921101    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:35.020605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.020564    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.020605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.020613    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.020888 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.020673    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.020888 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.020726    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.020888 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.020766    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.020888 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.020804    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8qfc\" (UniqueName: \"kubernetes.io/projected/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-kube-api-access-q8qfc\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.021518 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.021494    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.023399 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.023379    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.023399 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.023390    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.023785 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.023770    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.023882 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.023867    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.044067 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.044038    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8qfc\" (UniqueName: \"kubernetes.io/projected/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-kube-api-access-q8qfc\") pod \"data-science-collector-collector-0\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.213512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.213476    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:35.509370 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.509303    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:35.512503 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:27:35.512476    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2dabfd_4d60_41ec_ab5c_ae0d76a4fb48.slice/crio-5d3aefdf086892d59664fc72c4f2cac8665873c1b901dd9433155a5e56f1c1e6 WatchSource:0}: Error finding container 5d3aefdf086892d59664fc72c4f2cac8665873c1b901dd9433155a5e56f1c1e6: Status 404 returned error can't find the container with id 5d3aefdf086892d59664fc72c4f2cac8665873c1b901dd9433155a5e56f1c1e6
Oct 30 18:27:35.514729 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.514689    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:35.514836 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.514771    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:35.520810 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.520786    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"]
Oct 30 18:27:35.525086 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.525068    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.543817 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.543791    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"]
Oct 30 18:27:35.626360 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.626319    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mfrr\" (UniqueName: \"kubernetes.io/projected/a2709bca-42b8-4cb2-aab4-259f226c3d7f-kube-api-access-9mfrr\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.626360 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.626360    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/a2709bca-42b8-4cb2-aab4-259f226c3d7f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.626618 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.626386    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/a2709bca-42b8-4cb2-aab4-259f226c3d7f-ta-internal\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.727694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.727656    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mfrr\" (UniqueName: \"kubernetes.io/projected/a2709bca-42b8-4cb2-aab4-259f226c3d7f-kube-api-access-9mfrr\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.727866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.727708    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/a2709bca-42b8-4cb2-aab4-259f226c3d7f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.727866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.727736    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/a2709bca-42b8-4cb2-aab4-259f226c3d7f-ta-internal\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.728407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.728385    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/a2709bca-42b8-4cb2-aab4-259f226c3d7f-ta-internal\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.730332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.730305    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/a2709bca-42b8-4cb2-aab4-259f226c3d7f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.751146 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.751115    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mfrr\" (UniqueName: \"kubernetes.io/projected/a2709bca-42b8-4cb2-aab4-259f226c3d7f-kube-api-access-9mfrr\") pod \"data-science-collector-targetallocator-67d555679-5q474\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:35.806524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.806428    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48","Type":"ContainerStarted","Data":"5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb"}
Oct 30 18:27:35.806524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.806490    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48","Type":"ContainerStarted","Data":"5d3aefdf086892d59664fc72c4f2cac8665873c1b901dd9433155a5e56f1c1e6"}
Oct 30 18:27:35.849145 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.849086    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.849070358 podStartE2EDuration="1.849070358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:27:35.848381692 +0000 UTC m=+1826.015199820" watchObservedRunningTime="2026-10-30 18:27:35.849070358 +0000 UTC m=+1826.015888485"
Oct 30 18:27:35.859695 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:35.859646    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:36.012436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:36.012402    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72f2cf56-9058-45b2-8c66-b604d25e388e" path="/var/lib/kubelet/pods/72f2cf56-9058-45b2-8c66-b604d25e388e/volumes"
Oct 30 18:27:36.164233 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:36.164201    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"]
Oct 30 18:27:36.167212 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:27:36.167180    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2709bca_42b8_4cb2_aab4_259f226c3d7f.slice/crio-ce01b4bed001df59724999116ce5425539eca792807561d73cc9081debb33cb0 WatchSource:0}: Error finding container ce01b4bed001df59724999116ce5425539eca792807561d73cc9081debb33cb0: Status 404 returned error can't find the container with id ce01b4bed001df59724999116ce5425539eca792807561d73cc9081debb33cb0
Oct 30 18:27:36.169319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:36.169285    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:36.169406 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:36.169343    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:36.813424 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:36.813388    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474" event={"ID":"a2709bca-42b8-4cb2-aab4-259f226c3d7f","Type":"ContainerStarted","Data":"005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e"}
Oct 30 18:27:36.813424 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:36.813431    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474" event={"ID":"a2709bca-42b8-4cb2-aab4-259f226c3d7f","Type":"ContainerStarted","Data":"ce01b4bed001df59724999116ce5425539eca792807561d73cc9081debb33cb0"}
Oct 30 18:27:36.855173 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:36.855125    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474" podStartSLOduration=1.855111033 podStartE2EDuration="1.855111033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:27:36.854641483 +0000 UTC m=+1827.021459613" watchObservedRunningTime="2026-10-30 18:27:36.855111033 +0000 UTC m=+1827.021929160"
Oct 30 18:27:37.818931 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:37.818882    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:37.820797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:37.820777    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:37.890958 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:37.890920    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"]
Oct 30 18:27:37.891104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:37.891090    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl" podUID="1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" containerName="ta-container" containerID="cri-o://edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c" gracePeriod=30
Oct 30 18:27:38.135258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.135232    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:38.150184 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.150157    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-kube-api-access-v79vh\" (UniqueName: \"kubernetes.io/projected/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-kube-api-access-v79vh\") pod \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") "
Oct 30 18:27:38.150328 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.150241    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-ta-internal\" (UniqueName: \"kubernetes.io/configmap/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-ta-internal\") pod \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") "
Oct 30 18:27:38.150328 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.150273    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-data-science-collector-ta-server-cert\") pod \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\" (UID: \"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f\") "
Oct 30 18:27:38.150614 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.150586    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-ta-internal" pod "1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" (UID: "1bc2ec1c-3604-4515-ba9a-f28aafafdc9f"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:27:38.152424 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.152401    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-data-science-collector-ta-server-cert" pod "1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" (UID: "1bc2ec1c-3604-4515-ba9a-f28aafafdc9f"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:38.152528 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.152510    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-kube-api-access-v79vh" pod "1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" (UID: "1bc2ec1c-3604-4515-ba9a-f28aafafdc9f"). InnerVolumeSpecName "kube-api-access-v79vh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:27:38.251642 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.251610    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:38.251642 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.251637    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:38.251642 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.251647    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v79vh\" (UniqueName: \"kubernetes.io/projected/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f-kube-api-access-v79vh\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:38.824983 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.824943    5746 generic.go:356] "Generic (PLEG): container finished" podID="1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" containerID="edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c" exitCode=0
Oct 30 18:27:38.825521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.824997    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"
Oct 30 18:27:38.825521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.825035    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl" event={"ID":"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f","Type":"ContainerDied","Data":"edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c"}
Oct 30 18:27:38.825521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.825082    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl" event={"ID":"1bc2ec1c-3604-4515-ba9a-f28aafafdc9f","Type":"ContainerDied","Data":"11ec6e378f4915cd0e7794333b1b16ffd35711f7dccf0497762888272428afee"}
Oct 30 18:27:38.825521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.825100    5746 scope.go:122] "RemoveContainer" containerID="edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c"
Oct 30 18:27:38.834275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.834255    5746 scope.go:122] "RemoveContainer" containerID="edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c"
Oct 30 18:27:38.834536 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:27:38.834519    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c\": container with ID starting with edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c not found: ID does not exist" containerID="edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c"
Oct 30 18:27:38.834575 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.834546    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c"} err="failed to get container status \"edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c\": rpc error: code = NotFound desc = could not find container \"edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c\": container with ID starting with edbbf28fdb2c7ae6ce44234704a9dd3dc3e771e99aaf6d71dc13413ea08f9f1c not found: ID does not exist"
Oct 30 18:27:38.874773 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.874739    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"]
Oct 30 18:27:38.886947 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:38.886897    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54b49c7998-wxntl"]
Oct 30 18:27:40.008289 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:40.008250    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" path="/var/lib/kubelet/pods/1bc2ec1c-3604-4515-ba9a-f28aafafdc9f/volumes"
Oct 30 18:27:42.790340 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:27:42.790306    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:27:52.496752 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.496715    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"]
Oct 30 18:27:52.497159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.497062    5746 cpu_manager.go:414] "containerMap: removing container" podUID="1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" containerName="ta-container"
Oct 30 18:27:52.497159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.497074    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" containerName="ta-container"
Oct 30 18:27:52.497159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.497128    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bc2ec1c-3604-4515-ba9a-f28aafafdc9f" containerName="ta-container"
Oct 30 18:27:52.500996 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.500982    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.520559 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.520531    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"]
Oct 30 18:27:52.577700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.577669    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6qz7\" (UniqueName: \"kubernetes.io/projected/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-kube-api-access-r6qz7\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.577700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.577702    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.577959 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.577845    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-ta-internal\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.679089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.679056    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-ta-internal\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.679251 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.679101    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6qz7\" (UniqueName: \"kubernetes.io/projected/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-kube-api-access-r6qz7\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.679251 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.679121    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.679762 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.679731    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-ta-internal\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.681488 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.681463    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.702202 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.702177    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6qz7\" (UniqueName: \"kubernetes.io/projected/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-kube-api-access-r6qz7\") pod \"data-science-collector-targetallocator-5dd6b64cb6-cjggb\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:52.812284 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:52.812211    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:53.106838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:53.106802    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"]
Oct 30 18:27:53.110012 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:27:53.109983    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b6e35fa_0df7_45d5_99e1_8b3686a7e174.slice/crio-e019faadb034b3400dfb17f24bd67b7c7626858279e6cf987430bd22ed980be4 WatchSource:0}: Error finding container e019faadb034b3400dfb17f24bd67b7c7626858279e6cf987430bd22ed980be4: Status 404 returned error can't find the container with id e019faadb034b3400dfb17f24bd67b7c7626858279e6cf987430bd22ed980be4
Oct 30 18:27:53.112183 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:53.112149    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:53.112286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:53.112218    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:53.891299 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:53.891266    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb" event={"ID":"2b6e35fa-0df7-45d5-99e1-8b3686a7e174","Type":"ContainerStarted","Data":"d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10"}
Oct 30 18:27:53.891783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:53.891308    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb" event={"ID":"2b6e35fa-0df7-45d5-99e1-8b3686a7e174","Type":"ContainerStarted","Data":"e019faadb034b3400dfb17f24bd67b7c7626858279e6cf987430bd22ed980be4"}
Oct 30 18:27:53.891783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:53.891419    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:53.893037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:53.893014    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:27:53.932116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:53.932065    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb" podStartSLOduration=1.9320518519999998 podStartE2EDuration="1.932051852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:27:53.931667477 +0000 UTC m=+1844.098485617" watchObservedRunningTime="2026-10-30 18:27:53.932051852 +0000 UTC m=+1844.098869970"
Oct 30 18:27:54.001659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.001625    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"]
Oct 30 18:27:54.001827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.001803    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474" podUID="a2709bca-42b8-4cb2-aab4-259f226c3d7f" containerName="ta-container" containerID="cri-o://005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e" gracePeriod=30
Oct 30 18:27:54.235960 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.235934    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:54.293538 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.293493    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2709bca-42b8-4cb2-aab4-259f226c3d7f-ta-internal\" (UniqueName: \"kubernetes.io/configmap/a2709bca-42b8-4cb2-aab4-259f226c3d7f-ta-internal\") pod \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") "
Oct 30 18:27:54.293722 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.293590    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2709bca-42b8-4cb2-aab4-259f226c3d7f-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/a2709bca-42b8-4cb2-aab4-259f226c3d7f-data-science-collector-ta-server-cert\") pod \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") "
Oct 30 18:27:54.293722 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.293662    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2709bca-42b8-4cb2-aab4-259f226c3d7f-kube-api-access-9mfrr\" (UniqueName: \"kubernetes.io/projected/a2709bca-42b8-4cb2-aab4-259f226c3d7f-kube-api-access-9mfrr\") pod \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\" (UID: \"a2709bca-42b8-4cb2-aab4-259f226c3d7f\") "
Oct 30 18:27:54.293863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.293838    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2709bca-42b8-4cb2-aab4-259f226c3d7f-ta-internal" pod "a2709bca-42b8-4cb2-aab4-259f226c3d7f" (UID: "a2709bca-42b8-4cb2-aab4-259f226c3d7f"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:27:54.295854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.295821    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2709bca-42b8-4cb2-aab4-259f226c3d7f-data-science-collector-ta-server-cert" pod "a2709bca-42b8-4cb2-aab4-259f226c3d7f" (UID: "a2709bca-42b8-4cb2-aab4-259f226c3d7f"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:54.295854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.295830    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2709bca-42b8-4cb2-aab4-259f226c3d7f-kube-api-access-9mfrr" pod "a2709bca-42b8-4cb2-aab4-259f226c3d7f" (UID: "a2709bca-42b8-4cb2-aab4-259f226c3d7f"). InnerVolumeSpecName "kube-api-access-9mfrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:27:54.394867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.394835    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mfrr\" (UniqueName: \"kubernetes.io/projected/a2709bca-42b8-4cb2-aab4-259f226c3d7f-kube-api-access-9mfrr\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:54.394867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.394861    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/a2709bca-42b8-4cb2-aab4-259f226c3d7f-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:54.394867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.394871    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/a2709bca-42b8-4cb2-aab4-259f226c3d7f-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:54.895834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.895795    5746 generic.go:356] "Generic (PLEG): container finished" podID="a2709bca-42b8-4cb2-aab4-259f226c3d7f" containerID="005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e" exitCode=0
Oct 30 18:27:54.895834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.895838    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"
Oct 30 18:27:54.896377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.895881    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474" event={"ID":"a2709bca-42b8-4cb2-aab4-259f226c3d7f","Type":"ContainerDied","Data":"005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e"}
Oct 30 18:27:54.896377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.895946    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474" event={"ID":"a2709bca-42b8-4cb2-aab4-259f226c3d7f","Type":"ContainerDied","Data":"ce01b4bed001df59724999116ce5425539eca792807561d73cc9081debb33cb0"}
Oct 30 18:27:54.896377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.895972    5746 scope.go:122] "RemoveContainer" containerID="005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e"
Oct 30 18:27:54.904708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.904690    5746 scope.go:122] "RemoveContainer" containerID="005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e"
Oct 30 18:27:54.904993 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:27:54.904974    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e\": container with ID starting with 005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e not found: ID does not exist" containerID="005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e"
Oct 30 18:27:54.905055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.905001    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e"} err="failed to get container status \"005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e\": rpc error: code = NotFound desc = could not find container \"005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e\": container with ID starting with 005b9f99364a2496bc40134d233a0e06c0d2d2769878c2c7ebf19791a3dc548e not found: ID does not exist"
Oct 30 18:27:54.945769 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.945740    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"]
Oct 30 18:27:54.957789 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:54.957764    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-67d555679-5q474"]
Oct 30 18:27:56.012513 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:56.012477    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2709bca-42b8-4cb2-aab4-259f226c3d7f" path="/var/lib/kubelet/pods/a2709bca-42b8-4cb2-aab4-259f226c3d7f/volumes"
Oct 30 18:27:56.020622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:56.020592    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:56.020827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:56.020805    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" containerName="otc-container" containerID="cri-o://5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb" gracePeriod=30
Oct 30 18:27:57.263512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.263487    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:57.323112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.323082    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-kube-api-access-q8qfc\" (UniqueName: \"kubernetes.io/projected/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-kube-api-access-q8qfc\") pod \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") "
Oct 30 18:27:57.323272 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.323122    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otlp-tls-certs\") pod \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") "
Oct 30 18:27:57.323272 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.323144    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-data-science-collector-ta-client-cert\") pod \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") "
Oct 30 18:27:57.323272 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.323169    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-tls-certs\") pod \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") "
Oct 30 18:27:57.323499 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.323347    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otc-internal\" (UniqueName: \"kubernetes.io/configmap/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otc-internal\") pod \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") "
Oct 30 18:27:57.323499 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.323390    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-monitoring-tls-certs\") pod \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\" (UID: \"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48\") "
Oct 30 18:27:57.323808 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.323754    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otc-internal" pod "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" (UID: "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:27:57.325707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.325669    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-tls-certs" pod "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" (UID: "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:57.326012 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.325983    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-monitoring-tls-certs" pod "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" (UID: "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:57.326106 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.326007    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-kube-api-access-q8qfc" pod "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" (UID: "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48"). InnerVolumeSpecName "kube-api-access-q8qfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:27:57.326106 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.326018    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otlp-tls-certs" pod "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" (UID: "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:57.326106 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.326065    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-data-science-collector-ta-client-cert" pod "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" (UID: "ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:27:57.424725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.424688    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8qfc\" (UniqueName: \"kubernetes.io/projected/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-kube-api-access-q8qfc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:57.424725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.424724    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:57.424725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.424735    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:57.424893 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.424744    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:57.424893 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.424753    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:57.424893 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.424761    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:27:57.911331 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.911296    5746 generic.go:356] "Generic (PLEG): container finished" podID="ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" containerID="5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb" exitCode=0
Oct 30 18:27:57.911531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.911342    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:57.911531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.911362    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48","Type":"ContainerDied","Data":"5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb"}
Oct 30 18:27:57.911531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.911409    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48","Type":"ContainerDied","Data":"5d3aefdf086892d59664fc72c4f2cac8665873c1b901dd9433155a5e56f1c1e6"}
Oct 30 18:27:57.911531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.911433    5746 scope.go:122] "RemoveContainer" containerID="5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb"
Oct 30 18:27:57.920612 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.920595    5746 scope.go:122] "RemoveContainer" containerID="5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb"
Oct 30 18:27:57.920871 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:27:57.920856    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb\": container with ID starting with 5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb not found: ID does not exist" containerID="5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb"
Oct 30 18:27:57.920976 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.920880    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb"} err="failed to get container status \"5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb\": rpc error: code = NotFound desc = could not find container \"5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb\": container with ID starting with 5e37e8393c15db93085a968774ffee92096c31bdf9cda5316667caf76d64e5eb not found: ID does not exist"
Oct 30 18:27:57.961267 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.961246    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:57.973177 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:57.973152    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:58.009040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.009015    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" path="/var/lib/kubelet/pods/ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48/volumes"
Oct 30 18:27:58.009296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.009282    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:58.009561 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.009548    5746 cpu_manager.go:414] "containerMap: removing container" podUID="a2709bca-42b8-4cb2-aab4-259f226c3d7f" containerName="ta-container"
Oct 30 18:27:58.009599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.009564    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2709bca-42b8-4cb2-aab4-259f226c3d7f" containerName="ta-container"
Oct 30 18:27:58.009599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.009581    5746 cpu_manager.go:414] "containerMap: removing container" podUID="ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" containerName="otc-container"
Oct 30 18:27:58.009599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.009586    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" containerName="otc-container"
Oct 30 18:27:58.009693 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.009657    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2709bca-42b8-4cb2-aab4-259f226c3d7f" containerName="ta-container"
Oct 30 18:27:58.009693 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.009666    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea2dabfd-4d60-41ec-ab5c-ae0d76a4fb48" containerName="otc-container"
Oct 30 18:27:58.012507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.012493    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.029099 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.029045    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:27:58.029099 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.029090    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsdwv\""
Oct 30 18:27:58.029288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.029125    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-a0f9182a\""
Oct 30 18:27:58.029372 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.029344    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:27:58.029490 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.029398    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:27:58.030342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.030249    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:58.030690 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.030671    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:27:58.129824 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.129792    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.129824 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.129828    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.130097 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.129849    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.130097 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.130014    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrv22\" (UniqueName: \"kubernetes.io/projected/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-kube-api-access-vrv22\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.130097 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.130082    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.130216 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.130118    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.230850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.230774    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.230850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.230814    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.230850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.230843    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.231126 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.230895    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrv22\" (UniqueName: \"kubernetes.io/projected/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-kube-api-access-vrv22\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.231126 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.230950    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.231126 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.230982    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.231663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.231601    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otc-internal\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.233535 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.233510    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.233718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.233687    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.233867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.233852    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-monitoring-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.233945 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.233928    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otlp-tls-certs\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.254228 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.254205    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrv22\" (UniqueName: \"kubernetes.io/projected/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-kube-api-access-vrv22\") pod \"data-science-collector-collector-0\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") " pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.324689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.324659    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:27:58.621377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.621350    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:27:58.623948 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:27:58.623899    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd26ba27_a0dd_435d_a3d6_e58fc498ed83.slice/crio-a0a0d2842990af40bc4cc43a05829a58f932b9afffb878e7b6600dd047c565b4 WatchSource:0}: Error finding container a0a0d2842990af40bc4cc43a05829a58f932b9afffb878e7b6600dd047c565b4: Status 404 returned error can't find the container with id a0a0d2842990af40bc4cc43a05829a58f932b9afffb878e7b6600dd047c565b4
Oct 30 18:27:58.626056 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.626019    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:58.626172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.626093    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:27:58.920354 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.920261    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"cd26ba27-a0dd-435d-a3d6-e58fc498ed83","Type":"ContainerStarted","Data":"8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a"}
Oct 30 18:27:58.920354 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.920323    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"cd26ba27-a0dd-435d-a3d6-e58fc498ed83","Type":"ContainerStarted","Data":"a0a0d2842990af40bc4cc43a05829a58f932b9afffb878e7b6600dd047c565b4"}
Oct 30 18:27:58.965115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:27:58.965062    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-0" podStartSLOduration=1.965048412 podStartE2EDuration="1.965048412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:27:58.9642075 +0000 UTC m=+1849.131025651" watchObservedRunningTime="2026-10-30 18:27:58.965048412 +0000 UTC m=+1849.131866539"
Oct 30 18:28:00.618930 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.618879    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"]
Oct 30 18:28:00.621895 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.621877    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.650289 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.650259    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"]
Oct 30 18:28:00.752667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.752630    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/598412a1-b1a7-424b-83d9-61fe4ba57084-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.752850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.752710    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtgnz\" (UniqueName: \"kubernetes.io/projected/598412a1-b1a7-424b-83d9-61fe4ba57084-kube-api-access-xtgnz\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.752850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.752787    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/598412a1-b1a7-424b-83d9-61fe4ba57084-ta-internal\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.853389 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.853354    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/598412a1-b1a7-424b-83d9-61fe4ba57084-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.853569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.853423    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtgnz\" (UniqueName: \"kubernetes.io/projected/598412a1-b1a7-424b-83d9-61fe4ba57084-kube-api-access-xtgnz\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.853569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.853468    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/598412a1-b1a7-424b-83d9-61fe4ba57084-ta-internal\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.854172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.854146    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/598412a1-b1a7-424b-83d9-61fe4ba57084-ta-internal\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.855965 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.855946    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/598412a1-b1a7-424b-83d9-61fe4ba57084-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.876653 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.876594    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtgnz\" (UniqueName: \"kubernetes.io/projected/598412a1-b1a7-424b-83d9-61fe4ba57084-kube-api-access-xtgnz\") pod \"data-science-collector-targetallocator-7b486b6d77-462ck\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:00.933410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:00.933384    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:01.229622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:01.229533    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"]
Oct 30 18:28:01.233190 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:28:01.233164    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod598412a1_b1a7_424b_83d9_61fe4ba57084.slice/crio-61cddcae35138a7616f2db6dcdc4bd8fda8b2c24f570ab647333a50e82a53aaf WatchSource:0}: Error finding container 61cddcae35138a7616f2db6dcdc4bd8fda8b2c24f570ab647333a50e82a53aaf: Status 404 returned error can't find the container with id 61cddcae35138a7616f2db6dcdc4bd8fda8b2c24f570ab647333a50e82a53aaf
Oct 30 18:28:01.236029 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:01.235997    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:28:01.236101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:01.236060    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:28:01.933777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:01.933738    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck" event={"ID":"598412a1-b1a7-424b-83d9-61fe4ba57084","Type":"ContainerStarted","Data":"72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe"}
Oct 30 18:28:01.934259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:01.933785    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck" event={"ID":"598412a1-b1a7-424b-83d9-61fe4ba57084","Type":"ContainerStarted","Data":"61cddcae35138a7616f2db6dcdc4bd8fda8b2c24f570ab647333a50e82a53aaf"}
Oct 30 18:28:01.934259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:01.933884    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:01.935669 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:01.935649    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:28:02.030540 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.030486    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck" podStartSLOduration=2.030465028 podStartE2EDuration="2.030465028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:28:01.974617066 +0000 UTC m=+1852.141435194" watchObservedRunningTime="2026-10-30 18:28:02.030465028 +0000 UTC m=+1852.197283207"
Oct 30 18:28:02.086382 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.086353    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"]
Oct 30 18:28:02.086644 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.086614    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb" podUID="2b6e35fa-0df7-45d5-99e1-8b3686a7e174" containerName="ta-container" containerID="cri-o://d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10" gracePeriod=30
Oct 30 18:28:02.327463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.327436    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:28:02.469415 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.469337    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-ta-internal\" (UniqueName: \"kubernetes.io/configmap/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-ta-internal\") pod \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") "
Oct 30 18:28:02.469415 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.469384    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-kube-api-access-r6qz7\" (UniqueName: \"kubernetes.io/projected/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-kube-api-access-r6qz7\") pod \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") "
Oct 30 18:28:02.469607 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.469506    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-data-science-collector-ta-server-cert\") pod \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\" (UID: \"2b6e35fa-0df7-45d5-99e1-8b3686a7e174\") "
Oct 30 18:28:02.469749 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.469715    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-ta-internal" pod "2b6e35fa-0df7-45d5-99e1-8b3686a7e174" (UID: "2b6e35fa-0df7-45d5-99e1-8b3686a7e174"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:28:02.471661 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.471637    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-data-science-collector-ta-server-cert" pod "2b6e35fa-0df7-45d5-99e1-8b3686a7e174" (UID: "2b6e35fa-0df7-45d5-99e1-8b3686a7e174"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:28:02.471781 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.471760    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-kube-api-access-r6qz7" pod "2b6e35fa-0df7-45d5-99e1-8b3686a7e174" (UID: "2b6e35fa-0df7-45d5-99e1-8b3686a7e174"). InnerVolumeSpecName "kube-api-access-r6qz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:28:02.571013 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.570975    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:28:02.571013 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.571005    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:28:02.571013 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.571016    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6qz7\" (UniqueName: \"kubernetes.io/projected/2b6e35fa-0df7-45d5-99e1-8b3686a7e174-kube-api-access-r6qz7\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:28:02.938790 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.938755    5746 generic.go:356] "Generic (PLEG): container finished" podID="2b6e35fa-0df7-45d5-99e1-8b3686a7e174" containerID="d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10" exitCode=0
Oct 30 18:28:02.939334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.938803    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"
Oct 30 18:28:02.939334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.938834    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb" event={"ID":"2b6e35fa-0df7-45d5-99e1-8b3686a7e174","Type":"ContainerDied","Data":"d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10"}
Oct 30 18:28:02.939334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.938866    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb" event={"ID":"2b6e35fa-0df7-45d5-99e1-8b3686a7e174","Type":"ContainerDied","Data":"e019faadb034b3400dfb17f24bd67b7c7626858279e6cf987430bd22ed980be4"}
Oct 30 18:28:02.939334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.938885    5746 scope.go:122] "RemoveContainer" containerID="d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10"
Oct 30 18:28:02.947395 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.947374    5746 scope.go:122] "RemoveContainer" containerID="d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10"
Oct 30 18:28:02.947654 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:28:02.947626    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10\": container with ID starting with d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10 not found: ID does not exist" containerID="d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10"
Oct 30 18:28:02.947750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.947665    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10"} err="failed to get container status \"d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10\": rpc error: code = NotFound desc = could not find container \"d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10\": container with ID starting with d6a2b74ae08bd7590eb9cf53d5b2970667ef7c0623a39772e2434d536bc08c10 not found: ID does not exist"
Oct 30 18:28:02.988988 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:02.988958    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"]
Oct 30 18:28:03.001022 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:03.000994    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-cjggb"]
Oct 30 18:28:04.009176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:28:04.009142    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b6e35fa-0df7-45d5-99e1-8b3686a7e174" path="/var/lib/kubelet/pods/2b6e35fa-0df7-45d5-99e1-8b3686a7e174/volumes"
Oct 30 18:28:12.789608 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:28:12.789573    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:28:42.793135 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:28:42.793090    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:29:09.568152 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.568120    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["tests-monitoring-otlp-identity/otlp-identity-spoof-probe"]
Oct 30 18:29:09.568568 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.568407    5746 cpu_manager.go:414] "containerMap: removing container" podUID="2b6e35fa-0df7-45d5-99e1-8b3686a7e174" containerName="ta-container"
Oct 30 18:29:09.568568 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.568418    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b6e35fa-0df7-45d5-99e1-8b3686a7e174" containerName="ta-container"
Oct 30 18:29:09.568568 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.568481    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b6e35fa-0df7-45d5-99e1-8b3686a7e174" containerName="ta-container"
Oct 30 18:29:09.571385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.571367    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe"
Oct 30 18:29:09.585014 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.584994    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f278a0-504a-479f-a4ae-d25286aff8d3-service-ca\") pod \"otlp-identity-spoof-probe\" (UID: \"a2f278a0-504a-479f-a4ae-d25286aff8d3\") " pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe"
Oct 30 18:29:09.588329 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.588309    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"tests-monitoring-otlp-identity\"/\"openshift-service-ca.crt\""
Oct 30 18:29:09.589083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.589061    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["tests-monitoring-otlp-identity/otlp-identity-spoof-probe"]
Oct 30 18:29:09.600155 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.600136    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"tests-monitoring-otlp-identity\"/\"default-dockercfg-wp6wx\""
Oct 30 18:29:09.686353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.686322    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f278a0-504a-479f-a4ae-d25286aff8d3-service-ca\") pod \"otlp-identity-spoof-probe\" (UID: \"a2f278a0-504a-479f-a4ae-d25286aff8d3\") " pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe"
Oct 30 18:29:09.686939 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.686898    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f278a0-504a-479f-a4ae-d25286aff8d3-service-ca\") pod \"otlp-identity-spoof-probe\" (UID: \"a2f278a0-504a-479f-a4ae-d25286aff8d3\") " pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe"
Oct 30 18:29:09.882342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:09.882309    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe"
Oct 30 18:29:10.174672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:10.174604    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["tests-monitoring-otlp-identity/otlp-identity-spoof-probe"]
Oct 30 18:29:10.178187 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:29:10.178162    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2f278a0_504a_479f_a4ae_d25286aff8d3.slice/crio-76ff9b354e59a876f9e5163dba8a6d81c5992d864398de577a2ba06be2663137 WatchSource:0}: Error finding container 76ff9b354e59a876f9e5163dba8a6d81c5992d864398de577a2ba06be2663137: Status 404 returned error can't find the container with id 76ff9b354e59a876f9e5163dba8a6d81c5992d864398de577a2ba06be2663137
Oct 30 18:29:10.194736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:10.194703    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe" event={"ID":"a2f278a0-504a-479f-a4ae-d25286aff8d3","Type":"ContainerStarted","Data":"76ff9b354e59a876f9e5163dba8a6d81c5992d864398de577a2ba06be2663137"}
Oct 30 18:29:11.200964 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:11.200928    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe" event={"ID":"a2f278a0-504a-479f-a4ae-d25286aff8d3","Type":"ContainerStarted","Data":"5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c"}
Oct 30 18:29:11.240148 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:29:11.240094    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe" podStartSLOduration=2.240081946 podStartE2EDuration="2.240081946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:29:11.23941557 +0000 UTC m=+1921.406233709" watchObservedRunningTime="2026-10-30 18:29:11.240081946 +0000 UTC m=+1921.406900070"
Oct 30 18:29:12.790150 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:29:12.790115    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:29:42.790248 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:29:42.790210    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:30:12.791710 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:30:12.791674    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:30:42.790015 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:30:42.789933    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:31:12.790075 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:31:12.790041    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:31:42.789812 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:31:42.789780    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:31:52.269199 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:31:52.269164    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["tests-monitoring-otlp-identity/otlp-identity-spoof-probe"]
Oct 30 18:31:52.269622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:31:52.269363    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe" podUID="a2f278a0-504a-479f-a4ae-d25286aff8d3" containerName="otlp-spoof-probe" containerID="cri-o://5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c" gracePeriod=30
Oct 30 18:32:12.789145 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:32:12.789111    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:32:22.408427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.408403    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe"
Oct 30 18:32:22.527155 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.527068    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2f278a0-504a-479f-a4ae-d25286aff8d3-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f278a0-504a-479f-a4ae-d25286aff8d3-service-ca\") pod \"a2f278a0-504a-479f-a4ae-d25286aff8d3\" (UID: \"a2f278a0-504a-479f-a4ae-d25286aff8d3\") "
Oct 30 18:32:22.527427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.527403    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f278a0-504a-479f-a4ae-d25286aff8d3-service-ca" pod "a2f278a0-504a-479f-a4ae-d25286aff8d3" (UID: "a2f278a0-504a-479f-a4ae-d25286aff8d3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:32:22.628521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.628492    5746 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f278a0-504a-479f-a4ae-d25286aff8d3-service-ca\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:22.928244 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.928214    5746 generic.go:356] "Generic (PLEG): container finished" podID="a2f278a0-504a-479f-a4ae-d25286aff8d3" containerID="5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c" exitCode=137
Oct 30 18:32:22.928452 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.928263    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe"
Oct 30 18:32:22.928452 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.928270    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe" event={"ID":"a2f278a0-504a-479f-a4ae-d25286aff8d3","Type":"ContainerDied","Data":"5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c"}
Oct 30 18:32:22.928452 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.928303    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="tests-monitoring-otlp-identity/otlp-identity-spoof-probe" event={"ID":"a2f278a0-504a-479f-a4ae-d25286aff8d3","Type":"ContainerDied","Data":"76ff9b354e59a876f9e5163dba8a6d81c5992d864398de577a2ba06be2663137"}
Oct 30 18:32:22.928452 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.928326    5746 scope.go:122] "RemoveContainer" containerID="5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c"
Oct 30 18:32:22.936886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.936873    5746 scope.go:122] "RemoveContainer" containerID="5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c"
Oct 30 18:32:22.937144 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:32:22.937125    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c\": container with ID starting with 5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c not found: ID does not exist" containerID="5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c"
Oct 30 18:32:22.937190 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.937153    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c"} err="failed to get container status \"5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c\": rpc error: code = NotFound desc = could not find container \"5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c\": container with ID starting with 5d25ed802e1742e078ff1f2e3d3e93c06986267888cd158528ba75b3b753112c not found: ID does not exist"
Oct 30 18:32:22.978118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.978095    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["tests-monitoring-otlp-identity/otlp-identity-spoof-probe"]
Oct 30 18:32:22.984782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:22.984755    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["tests-monitoring-otlp-identity/otlp-identity-spoof-probe"]
Oct 30 18:32:24.007799 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:24.007768    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2f278a0-504a-479f-a4ae-d25286aff8d3" path="/var/lib/kubelet/pods/a2f278a0-504a-479f-a4ae-d25286aff8d3/volumes"
Oct 30 18:32:31.639623 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.639591    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"]
Oct 30 18:32:31.640065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.639936    5746 cpu_manager.go:414] "containerMap: removing container" podUID="a2f278a0-504a-479f-a4ae-d25286aff8d3" containerName="otlp-spoof-probe"
Oct 30 18:32:31.640065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.639948    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2f278a0-504a-479f-a4ae-d25286aff8d3" containerName="otlp-spoof-probe"
Oct 30 18:32:31.640065 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.640010    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2f278a0-504a-479f-a4ae-d25286aff8d3" containerName="otlp-spoof-probe"
Oct 30 18:32:31.643137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.643118    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.672595 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.672569    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"]
Oct 30 18:32:31.703370 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.703349    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl6j8\" (UniqueName: \"kubernetes.io/projected/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-kube-api-access-kl6j8\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.703493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.703381    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.703493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.703401    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-ta-internal\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.804273 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.804239    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-ta-internal\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.804435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.804354    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl6j8\" (UniqueName: \"kubernetes.io/projected/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-kube-api-access-kl6j8\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.804435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.804394    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.804865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.804843    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-ta-internal\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.806939 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.806898    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.828046 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.828025    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl6j8\" (UniqueName: \"kubernetes.io/projected/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-kube-api-access-kl6j8\") pod \"data-science-collector-targetallocator-766cff4576-kz6nx\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:31.954075 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:31.954005    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:32.250049 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:32.249977    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"]
Oct 30 18:32:32.253173 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:32:32.253143    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddec06f6a_bff6_45a0_ab84_0a2c11881f4c.slice/crio-3308a909141a91d87454f47c4a7f51684d77a3e88e14ad6424ead224978c2333 WatchSource:0}: Error finding container 3308a909141a91d87454f47c4a7f51684d77a3e88e14ad6424ead224978c2333: Status 404 returned error can't find the container with id 3308a909141a91d87454f47c4a7f51684d77a3e88e14ad6424ead224978c2333
Oct 30 18:32:32.255764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:32.255733    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:32:32.255824 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:32.255794    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:32:32.968461 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:32.968417    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx" event={"ID":"dec06f6a-bff6-45a0-ab84-0a2c11881f4c","Type":"ContainerStarted","Data":"db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9"}
Oct 30 18:32:32.968940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:32.968468    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx" event={"ID":"dec06f6a-bff6-45a0-ab84-0a2c11881f4c","Type":"ContainerStarted","Data":"3308a909141a91d87454f47c4a7f51684d77a3e88e14ad6424ead224978c2333"}
Oct 30 18:32:32.968940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:32.968586    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:32.970351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:32.970330    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:33.011154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.011096    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx" podStartSLOduration=2.0110841 podStartE2EDuration="2.0110841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:33.010166172 +0000 UTC m=+2123.176984316" watchObservedRunningTime="2026-10-30 18:32:33.0110841 +0000 UTC m=+2123.177902227"
Oct 30 18:32:33.083759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.083732    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"]
Oct 30 18:32:33.083938 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.083898    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck" podUID="598412a1-b1a7-424b-83d9-61fe4ba57084" containerName="ta-container" containerID="cri-o://72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe" gracePeriod=30
Oct 30 18:32:33.333623 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.333600    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:32:33.419263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.419235    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/598412a1-b1a7-424b-83d9-61fe4ba57084-ta-internal\" (UniqueName: \"kubernetes.io/configmap/598412a1-b1a7-424b-83d9-61fe4ba57084-ta-internal\") pod \"598412a1-b1a7-424b-83d9-61fe4ba57084\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") "
Oct 30 18:32:33.419432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.419341    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/598412a1-b1a7-424b-83d9-61fe4ba57084-kube-api-access-xtgnz\" (UniqueName: \"kubernetes.io/projected/598412a1-b1a7-424b-83d9-61fe4ba57084-kube-api-access-xtgnz\") pod \"598412a1-b1a7-424b-83d9-61fe4ba57084\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") "
Oct 30 18:32:33.419432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.419378    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/598412a1-b1a7-424b-83d9-61fe4ba57084-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/598412a1-b1a7-424b-83d9-61fe4ba57084-data-science-collector-ta-server-cert\") pod \"598412a1-b1a7-424b-83d9-61fe4ba57084\" (UID: \"598412a1-b1a7-424b-83d9-61fe4ba57084\") "
Oct 30 18:32:33.419655 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.419630    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598412a1-b1a7-424b-83d9-61fe4ba57084-ta-internal" pod "598412a1-b1a7-424b-83d9-61fe4ba57084" (UID: "598412a1-b1a7-424b-83d9-61fe4ba57084"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:32:33.421501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.421473    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598412a1-b1a7-424b-83d9-61fe4ba57084-kube-api-access-xtgnz" pod "598412a1-b1a7-424b-83d9-61fe4ba57084" (UID: "598412a1-b1a7-424b-83d9-61fe4ba57084"). InnerVolumeSpecName "kube-api-access-xtgnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:32:33.421604 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.421547    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/598412a1-b1a7-424b-83d9-61fe4ba57084-data-science-collector-ta-server-cert" pod "598412a1-b1a7-424b-83d9-61fe4ba57084" (UID: "598412a1-b1a7-424b-83d9-61fe4ba57084"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:32:33.520256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.520194    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtgnz\" (UniqueName: \"kubernetes.io/projected/598412a1-b1a7-424b-83d9-61fe4ba57084-kube-api-access-xtgnz\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:33.520256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.520219    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/598412a1-b1a7-424b-83d9-61fe4ba57084-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:33.520256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.520231    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/598412a1-b1a7-424b-83d9-61fe4ba57084-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:33.973931 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.973878    5746 generic.go:356] "Generic (PLEG): container finished" podID="598412a1-b1a7-424b-83d9-61fe4ba57084" containerID="72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe" exitCode=0
Oct 30 18:32:33.974410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.973944    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"
Oct 30 18:32:33.974410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.973965    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck" event={"ID":"598412a1-b1a7-424b-83d9-61fe4ba57084","Type":"ContainerDied","Data":"72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe"}
Oct 30 18:32:33.974410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.974006    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck" event={"ID":"598412a1-b1a7-424b-83d9-61fe4ba57084","Type":"ContainerDied","Data":"61cddcae35138a7616f2db6dcdc4bd8fda8b2c24f570ab647333a50e82a53aaf"}
Oct 30 18:32:33.974410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.974021    5746 scope.go:122] "RemoveContainer" containerID="72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe"
Oct 30 18:32:33.984215 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.984135    5746 scope.go:122] "RemoveContainer" containerID="72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe"
Oct 30 18:32:33.984487 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:32:33.984461    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe\": container with ID starting with 72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe not found: ID does not exist" containerID="72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe"
Oct 30 18:32:33.984597 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:33.984497    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe"} err="failed to get container status \"72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe\": rpc error: code = NotFound desc = could not find container \"72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe\": container with ID starting with 72f8850b40100cbd829280828dadceb166c821692125f09bb480b9a092af3dfe not found: ID does not exist"
Oct 30 18:32:34.025370 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:34.025346    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"]
Oct 30 18:32:34.037678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:34.037653    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b486b6d77-462ck"]
Oct 30 18:32:36.008003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:36.007964    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="598412a1-b1a7-424b-83d9-61fe4ba57084" path="/var/lib/kubelet/pods/598412a1-b1a7-424b-83d9-61fe4ba57084/volumes"
Oct 30 18:32:39.093122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.093079    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"]
Oct 30 18:32:39.093576 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.093403    5746 cpu_manager.go:414] "containerMap: removing container" podUID="598412a1-b1a7-424b-83d9-61fe4ba57084" containerName="ta-container"
Oct 30 18:32:39.093576 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.093417    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="598412a1-b1a7-424b-83d9-61fe4ba57084" containerName="ta-container"
Oct 30 18:32:39.093576 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.093475    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="598412a1-b1a7-424b-83d9-61fe4ba57084" containerName="ta-container"
Oct 30 18:32:39.098994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.098888    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.117806 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.117743    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"]
Oct 30 18:32:39.167745 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.167713    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.167956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.167778    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-ta-internal\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.168023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.167962    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94zgj\" (UniqueName: \"kubernetes.io/projected/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-kube-api-access-94zgj\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.268516 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.268489    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94zgj\" (UniqueName: \"kubernetes.io/projected/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-kube-api-access-94zgj\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.268700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.268538    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.268700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.268575    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-ta-internal\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.269220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.269200    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-ta-internal\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.271240 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.271211    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.291697 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.291672    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94zgj\" (UniqueName: \"kubernetes.io/projected/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-kube-api-access-94zgj\") pod \"data-science-collector-targetallocator-5dd6b64cb6-sj2wt\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.410655 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.410621    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:39.706507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.706434    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"]
Oct 30 18:32:39.709356 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:32:39.709322    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61b3cc2d_0264_4a2c_9c3a_58ae70fcbd0c.slice/crio-3fe0339094df455f10842a26cba989bc21badc46129235917e5dec8e68dc0089 WatchSource:0}: Error finding container 3fe0339094df455f10842a26cba989bc21badc46129235917e5dec8e68dc0089: Status 404 returned error can't find the container with id 3fe0339094df455f10842a26cba989bc21badc46129235917e5dec8e68dc0089
Oct 30 18:32:39.711481 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.711445    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:32:39.711594 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.711543    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:32:39.999570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.999455    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt" event={"ID":"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c","Type":"ContainerStarted","Data":"1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645"}
Oct 30 18:32:39.999570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.999497    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt" event={"ID":"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c","Type":"ContainerStarted","Data":"3fe0339094df455f10842a26cba989bc21badc46129235917e5dec8e68dc0089"}
Oct 30 18:32:39.999754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:39.999616    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:40.000510 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:40.000487    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt" podUID="61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.67:8080/readyz\": dial tcp 10.133.0.67:8080: connect: connection refused"
Oct 30 18:32:40.042379 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:40.042324    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt" podStartSLOduration=1.042312916 podStartE2EDuration="1.042312916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:40.041721508 +0000 UTC m=+2130.208539635" watchObservedRunningTime="2026-10-30 18:32:40.042312916 +0000 UTC m=+2130.209131061"
Oct 30 18:32:41.004756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.004724    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:41.077024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.076992    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"]
Oct 30 18:32:41.077253 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.077197    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx" podUID="dec06f6a-bff6-45a0-ab84-0a2c11881f4c" containerName="ta-container" containerID="cri-o://db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9" gracePeriod=30
Oct 30 18:32:41.318311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.318281    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:41.389606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.389574    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-kube-api-access-kl6j8\" (UniqueName: \"kubernetes.io/projected/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-kube-api-access-kl6j8\") pod \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") "
Oct 30 18:32:41.389767 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.389620    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-data-science-collector-ta-server-cert\") pod \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") "
Oct 30 18:32:41.389767 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.389653    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-ta-internal\" (UniqueName: \"kubernetes.io/configmap/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-ta-internal\") pod \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\" (UID: \"dec06f6a-bff6-45a0-ab84-0a2c11881f4c\") "
Oct 30 18:32:41.390112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.390087    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-ta-internal" pod "dec06f6a-bff6-45a0-ab84-0a2c11881f4c" (UID: "dec06f6a-bff6-45a0-ab84-0a2c11881f4c"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:32:41.391807 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.391786    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-kube-api-access-kl6j8" pod "dec06f6a-bff6-45a0-ab84-0a2c11881f4c" (UID: "dec06f6a-bff6-45a0-ab84-0a2c11881f4c"). InnerVolumeSpecName "kube-api-access-kl6j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:32:41.391896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.391813    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-data-science-collector-ta-server-cert" pod "dec06f6a-bff6-45a0-ab84-0a2c11881f4c" (UID: "dec06f6a-bff6-45a0-ab84-0a2c11881f4c"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:32:41.490663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.490631    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:41.490663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.490658    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl6j8\" (UniqueName: \"kubernetes.io/projected/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-kube-api-access-kl6j8\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:41.490663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:41.490671    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/dec06f6a-bff6-45a0-ab84-0a2c11881f4c-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:42.008766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.008732    5746 generic.go:356] "Generic (PLEG): container finished" podID="dec06f6a-bff6-45a0-ab84-0a2c11881f4c" containerID="db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9" exitCode=0
Oct 30 18:32:42.009301 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.008788    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"
Oct 30 18:32:42.009301 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.008820    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx" event={"ID":"dec06f6a-bff6-45a0-ab84-0a2c11881f4c","Type":"ContainerDied","Data":"db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9"}
Oct 30 18:32:42.009301 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.008873    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx" event={"ID":"dec06f6a-bff6-45a0-ab84-0a2c11881f4c","Type":"ContainerDied","Data":"3308a909141a91d87454f47c4a7f51684d77a3e88e14ad6424ead224978c2333"}
Oct 30 18:32:42.009301 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.008902    5746 scope.go:122] "RemoveContainer" containerID="db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9"
Oct 30 18:32:42.017200 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.017182    5746 scope.go:122] "RemoveContainer" containerID="db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9"
Oct 30 18:32:42.017506 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:32:42.017484    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9\": container with ID starting with db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9 not found: ID does not exist" containerID="db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9"
Oct 30 18:32:42.017599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.017517    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9"} err="failed to get container status \"db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9\": rpc error: code = NotFound desc = could not find container \"db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9\": container with ID starting with db89b6021e3259f8a8c67e99b1d039f1ad6ca74b344f25525dd3a8d5b2d227a9 not found: ID does not exist"
Oct 30 18:32:42.058854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.058828    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"]
Oct 30 18:32:42.071025 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:42.070999    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-766cff4576-kz6nx"]
Oct 30 18:32:42.791066 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:32:42.791035    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:32:44.008237 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:44.008206    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dec06f6a-bff6-45a0-ab84-0a2c11881f4c" path="/var/lib/kubelet/pods/dec06f6a-bff6-45a0-ab84-0a2c11881f4c/volumes"
Oct 30 18:32:56.100646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.100604    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"]
Oct 30 18:32:56.101086 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.100923    5746 cpu_manager.go:414] "containerMap: removing container" podUID="dec06f6a-bff6-45a0-ab84-0a2c11881f4c" containerName="ta-container"
Oct 30 18:32:56.101086 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.100937    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dec06f6a-bff6-45a0-ab84-0a2c11881f4c" containerName="ta-container"
Oct 30 18:32:56.101086 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.101010    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="dec06f6a-bff6-45a0-ab84-0a2c11881f4c" containerName="ta-container"
Oct 30 18:32:56.106377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.106351    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.124896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.124866    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"]
Oct 30 18:32:56.217694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.217659    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ddabca8b-da35-4d7d-a04b-8a9af037dd57-ta-internal\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.217949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.217724    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ddabca8b-da35-4d7d-a04b-8a9af037dd57-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.217949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.217815    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlzm9\" (UniqueName: \"kubernetes.io/projected/ddabca8b-da35-4d7d-a04b-8a9af037dd57-kube-api-access-vlzm9\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.319055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.319021    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ddabca8b-da35-4d7d-a04b-8a9af037dd57-ta-internal\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.319248 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.319079    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ddabca8b-da35-4d7d-a04b-8a9af037dd57-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.319248 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.319110    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlzm9\" (UniqueName: \"kubernetes.io/projected/ddabca8b-da35-4d7d-a04b-8a9af037dd57-kube-api-access-vlzm9\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.319678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.319648    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ddabca8b-da35-4d7d-a04b-8a9af037dd57-ta-internal\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.321867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.321845    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ddabca8b-da35-4d7d-a04b-8a9af037dd57-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.342228 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.342198    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlzm9\" (UniqueName: \"kubernetes.io/projected/ddabca8b-da35-4d7d-a04b-8a9af037dd57-kube-api-access-vlzm9\") pod \"data-science-collector-targetallocator-54cd4cbf97-26746\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.419967 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.419884    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:56.715823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.715717    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"]
Oct 30 18:32:56.719710 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:32:56.719680    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddabca8b_da35_4d7d_a04b_8a9af037dd57.slice/crio-e43fad1a8339f5819e0e308b4267f0ad00cce04bc9e62950f293f8b5dfe3e92e WatchSource:0}: Error finding container e43fad1a8339f5819e0e308b4267f0ad00cce04bc9e62950f293f8b5dfe3e92e: Status 404 returned error can't find the container with id e43fad1a8339f5819e0e308b4267f0ad00cce04bc9e62950f293f8b5dfe3e92e
Oct 30 18:32:56.721971 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.721938    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:32:56.722078 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:56.722005    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:32:57.075409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:57.075301    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746" event={"ID":"ddabca8b-da35-4d7d-a04b-8a9af037dd57","Type":"ContainerStarted","Data":"02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd"}
Oct 30 18:32:57.075409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:57.075355    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746" event={"ID":"ddabca8b-da35-4d7d-a04b-8a9af037dd57","Type":"ContainerStarted","Data":"e43fad1a8339f5819e0e308b4267f0ad00cce04bc9e62950f293f8b5dfe3e92e"}
Oct 30 18:32:57.075409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:57.075376    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:57.076648 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:57.076620    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746" podUID="ddabca8b-da35-4d7d-a04b-8a9af037dd57" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.68:8080/readyz\": dial tcp 10.133.0.68:8080: connect: connection refused"
Oct 30 18:32:57.116626 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:57.116560    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746" podStartSLOduration=1.116538822 podStartE2EDuration="1.116538822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:57.115525765 +0000 UTC m=+2147.282343893" watchObservedRunningTime="2026-10-30 18:32:57.116538822 +0000 UTC m=+2147.283356950"
Oct 30 18:32:58.080623 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.080593    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:32:58.139154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.139126    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/operator-metrics-auth-probe"]
Oct 30 18:32:58.142427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.142406    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:32:58.150131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.150106    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"]
Oct 30 18:32:58.150348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.150321    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt" podUID="61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" containerName="ta-container" containerID="cri-o://1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645" gracePeriod=30
Oct 30 18:32:58.159004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.158982    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"odh-observability-metrics-ca\""
Oct 30 18:32:58.160115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.160094    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/operator-metrics-auth-probe"]
Oct 30 18:32:58.234673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.234639    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m2lm\" (UniqueName: \"kubernetes.io/projected/508d4f9d-680b-4e10-9897-4d89dbd6e747-kube-api-access-9m2lm\") pod \"operator-metrics-auth-probe\" (UID: \"508d4f9d-680b-4e10-9897-4d89dbd6e747\") " pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:32:58.234818 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.234708    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-ca\" (UniqueName: \"kubernetes.io/configmap/508d4f9d-680b-4e10-9897-4d89dbd6e747-metrics-ca\") pod \"operator-metrics-auth-probe\" (UID: \"508d4f9d-680b-4e10-9897-4d89dbd6e747\") " pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:32:58.335610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.335542    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m2lm\" (UniqueName: \"kubernetes.io/projected/508d4f9d-680b-4e10-9897-4d89dbd6e747-kube-api-access-9m2lm\") pod \"operator-metrics-auth-probe\" (UID: \"508d4f9d-680b-4e10-9897-4d89dbd6e747\") " pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:32:58.335727 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.335637    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-ca\" (UniqueName: \"kubernetes.io/configmap/508d4f9d-680b-4e10-9897-4d89dbd6e747-metrics-ca\") pod \"operator-metrics-auth-probe\" (UID: \"508d4f9d-680b-4e10-9897-4d89dbd6e747\") " pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:32:58.336284 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.336266    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-ca\" (UniqueName: \"kubernetes.io/configmap/508d4f9d-680b-4e10-9897-4d89dbd6e747-metrics-ca\") pod \"operator-metrics-auth-probe\" (UID: \"508d4f9d-680b-4e10-9897-4d89dbd6e747\") " pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:32:58.358843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.358818    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m2lm\" (UniqueName: \"kubernetes.io/projected/508d4f9d-680b-4e10-9897-4d89dbd6e747-kube-api-access-9m2lm\") pod \"operator-metrics-auth-probe\" (UID: \"508d4f9d-680b-4e10-9897-4d89dbd6e747\") " pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:32:58.397422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.397401    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:58.436138 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.436116    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-ta-internal\" (UniqueName: \"kubernetes.io/configmap/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-ta-internal\") pod \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") "
Oct 30 18:32:58.436271 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.436177    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-kube-api-access-94zgj\" (UniqueName: \"kubernetes.io/projected/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-kube-api-access-94zgj\") pod \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") "
Oct 30 18:32:58.436340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.436271    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-data-science-collector-ta-server-cert\") pod \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\" (UID: \"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c\") "
Oct 30 18:32:58.436476 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.436453    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-ta-internal" pod "61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" (UID: "61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:32:58.438855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.438819    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-data-science-collector-ta-server-cert" pod "61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" (UID: "61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:32:58.440704 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.440679    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-kube-api-access-94zgj" pod "61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" (UID: "61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c"). InnerVolumeSpecName "kube-api-access-94zgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:32:58.455260 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.455232    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:32:58.537008 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.536975    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94zgj\" (UniqueName: \"kubernetes.io/projected/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-kube-api-access-94zgj\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:58.537336 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.537013    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:58.537336 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.537029    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:32:58.747735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:58.747704    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/operator-metrics-auth-probe"]
Oct 30 18:32:58.750706 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:32:58.750668    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e WatchSource:0}: Error finding container 9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e: Status 404 returned error can't find the container with id 9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e
Oct 30 18:32:59.085219 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.085133    5746 generic.go:356] "Generic (PLEG): container finished" podID="61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" containerID="1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645" exitCode=0
Oct 30 18:32:59.085219 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.085175    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"
Oct 30 18:32:59.085454 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.085220    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt" event={"ID":"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c","Type":"ContainerDied","Data":"1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645"}
Oct 30 18:32:59.085454 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.085266    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt" event={"ID":"61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c","Type":"ContainerDied","Data":"3fe0339094df455f10842a26cba989bc21badc46129235917e5dec8e68dc0089"}
Oct 30 18:32:59.085454 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.085290    5746 scope.go:122] "RemoveContainer" containerID="1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645"
Oct 30 18:32:59.086980 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.086952    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/operator-metrics-auth-probe" event={"ID":"508d4f9d-680b-4e10-9897-4d89dbd6e747","Type":"ContainerStarted","Data":"8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457"}
Oct 30 18:32:59.087113 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.086996    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/operator-metrics-auth-probe" event={"ID":"508d4f9d-680b-4e10-9897-4d89dbd6e747","Type":"ContainerStarted","Data":"9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e"}
Oct 30 18:32:59.095111 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.095088    5746 scope.go:122] "RemoveContainer" containerID="1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645"
Oct 30 18:32:59.095408 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:32:59.095392    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645\": container with ID starting with 1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645 not found: ID does not exist" containerID="1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645"
Oct 30 18:32:59.095479 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.095416    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645"} err="failed to get container status \"1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645\": rpc error: code = NotFound desc = could not find container \"1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645\": container with ID starting with 1703bdd933c1a2b9f23f3181888b5acdd5730ad10f93c0ce4ce8bb84b4eb4645 not found: ID does not exist"
Oct 30 18:32:59.140210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.140151    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/operator-metrics-auth-probe" podStartSLOduration=1.14013767 podStartE2EDuration="1.14013767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:59.139690417 +0000 UTC m=+2149.306508563" watchObservedRunningTime="2026-10-30 18:32:59.14013767 +0000 UTC m=+2149.306955796"
Oct 30 18:32:59.174572 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.174544    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"]
Oct 30 18:32:59.178497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.178473    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5dd6b64cb6-sj2wt"]
Oct 30 18:32:59.459668 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.459634    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:32:59.459984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:32:59.459895    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-0" podUID="cd26ba27-a0dd-435d-a3d6-e58fc498ed83" containerName="otc-container" containerID="cri-o://8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a" gracePeriod=30
Oct 30 18:33:00.008020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:00.007981    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" path="/var/lib/kubelet/pods/61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c/volumes"
Oct 30 18:33:01.800365 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.800341    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:33:01.860558 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.860531    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-tls-certs\") pod \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") "
Oct 30 18:33:01.860727 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.860573    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-kube-api-access-vrv22\" (UniqueName: \"kubernetes.io/projected/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-kube-api-access-vrv22\") pod \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") "
Oct 30 18:33:01.860727 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.860609    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otlp-tls-certs\") pod \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") "
Oct 30 18:33:01.860857 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.860739    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otc-internal\" (UniqueName: \"kubernetes.io/configmap/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otc-internal\") pod \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") "
Oct 30 18:33:01.861207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.861180    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otc-internal" pod "cd26ba27-a0dd-435d-a3d6-e58fc498ed83" (UID: "cd26ba27-a0dd-435d-a3d6-e58fc498ed83"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:33:01.863013 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.862985    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-tls-certs" pod "cd26ba27-a0dd-435d-a3d6-e58fc498ed83" (UID: "cd26ba27-a0dd-435d-a3d6-e58fc498ed83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:33:01.863448 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.863428    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otlp-tls-certs" pod "cd26ba27-a0dd-435d-a3d6-e58fc498ed83" (UID: "cd26ba27-a0dd-435d-a3d6-e58fc498ed83"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:33:01.863448 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.863441    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-kube-api-access-vrv22" pod "cd26ba27-a0dd-435d-a3d6-e58fc498ed83" (UID: "cd26ba27-a0dd-435d-a3d6-e58fc498ed83"). InnerVolumeSpecName "kube-api-access-vrv22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:33:01.961373 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.961339    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-data-science-collector-ta-client-cert\") pod \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") "
Oct 30 18:33:01.961527 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.961381    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-monitoring-tls-certs\") pod \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\" (UID: \"cd26ba27-a0dd-435d-a3d6-e58fc498ed83\") "
Oct 30 18:33:01.961602 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.961580    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:01.961659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.961600    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrv22\" (UniqueName: \"kubernetes.io/projected/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-kube-api-access-vrv22\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:01.961659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.961614    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:01.961659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.961627    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:01.963456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.963434    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-data-science-collector-ta-client-cert" pod "cd26ba27-a0dd-435d-a3d6-e58fc498ed83" (UID: "cd26ba27-a0dd-435d-a3d6-e58fc498ed83"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:33:01.963521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:01.963470    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-monitoring-tls-certs" pod "cd26ba27-a0dd-435d-a3d6-e58fc498ed83" (UID: "cd26ba27-a0dd-435d-a3d6-e58fc498ed83"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:33:02.061997 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.061961    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:02.061997 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.061992    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/cd26ba27-a0dd-435d-a3d6-e58fc498ed83-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:02.101638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.101605    5746 generic.go:356] "Generic (PLEG): container finished" podID="cd26ba27-a0dd-435d-a3d6-e58fc498ed83" containerID="8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a" exitCode=0
Oct 30 18:33:02.101819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.101644    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-0"
Oct 30 18:33:02.101819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.101644    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"cd26ba27-a0dd-435d-a3d6-e58fc498ed83","Type":"ContainerDied","Data":"8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a"}
Oct 30 18:33:02.101956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.101804    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-0" event={"ID":"cd26ba27-a0dd-435d-a3d6-e58fc498ed83","Type":"ContainerDied","Data":"a0a0d2842990af40bc4cc43a05829a58f932b9afffb878e7b6600dd047c565b4"}
Oct 30 18:33:02.101956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.101841    5746 scope.go:122] "RemoveContainer" containerID="8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a"
Oct 30 18:33:02.110045 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.110022    5746 scope.go:122] "RemoveContainer" containerID="8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a"
Oct 30 18:33:02.110302 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:33:02.110286    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a\": container with ID starting with 8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a not found: ID does not exist" containerID="8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a"
Oct 30 18:33:02.110358 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.110310    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a"} err="failed to get container status \"8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a\": rpc error: code = NotFound desc = could not find container \"8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a\": container with ID starting with 8260ee936cc5db15f65b6ec6b62ca6f833b62b70d715370e749f41c771af444a not found: ID does not exist"
Oct 30 18:33:02.152214 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.152150    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:33:02.156152 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:02.156128    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-0"]
Oct 30 18:33:04.008040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:04.008009    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd26ba27-a0dd-435d-a3d6-e58fc498ed83" path="/var/lib/kubelet/pods/cd26ba27-a0dd-435d-a3d6-e58fc498ed83/volumes"
Oct 30 18:33:04.145042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:04.145011    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/operator-metrics-auth-probe"]
Oct 30 18:33:04.145229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:04.145178    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/operator-metrics-auth-probe" podUID="508d4f9d-680b-4e10-9897-4d89dbd6e747" containerName="curl" containerID="cri-o://8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457" gracePeriod=30
Oct 30 18:33:12.789366 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:33:12.789329    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:33:24.760826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.760746    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"]
Oct 30 18:33:24.761319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.761042    5746 cpu_manager.go:414] "containerMap: removing container" podUID="cd26ba27-a0dd-435d-a3d6-e58fc498ed83" containerName="otc-container"
Oct 30 18:33:24.761319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.761052    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd26ba27-a0dd-435d-a3d6-e58fc498ed83" containerName="otc-container"
Oct 30 18:33:24.761319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.761067    5746 cpu_manager.go:414] "containerMap: removing container" podUID="61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" containerName="ta-container"
Oct 30 18:33:24.761319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.761072    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" containerName="ta-container"
Oct 30 18:33:24.761319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.761126    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd26ba27-a0dd-435d-a3d6-e58fc498ed83" containerName="otc-container"
Oct 30 18:33:24.761319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.761139    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b3cc2d-0264-4a2c-9c3a-58ae70fcbd0c" containerName="ta-container"
Oct 30 18:33:24.765446 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.765429    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"
Oct 30 18:33:24.775532 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.775515    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"
Oct 30 18:33:24.781414 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:24.781393    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"]
Oct 30 18:33:25.073357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:25.073285    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"]
Oct 30 18:33:25.078894 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:25.078874    5746 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:33:25.201584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:25.201538    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977" event={"ID":"f34a344f-a4cb-442e-b035-9fe2d43ebe80","Type":"ContainerStarted","Data":"fdea6e193a43d1732ceec291e74c08b84474d343e4f636c868c4bf7233b26e3c"}
Oct 30 18:33:29.219941 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:29.219886    5746 generic.go:356] "Generic (PLEG): container finished" podID="f34a344f-a4cb-442e-b035-9fe2d43ebe80" containerID="d6cd46e515927df7abdea19fc6f48ea5ea501ff19696310d5ca72a8a309843bf" exitCode=0
Oct 30 18:33:29.220363 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:29.219975    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977" event={"ID":"f34a344f-a4cb-442e-b035-9fe2d43ebe80","Type":"ContainerDied","Data":"d6cd46e515927df7abdea19fc6f48ea5ea501ff19696310d5ca72a8a309843bf"}
Oct 30 18:33:30.349109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:30.349084    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"
Oct 30 18:33:31.228780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:31.228748    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977" event={"ID":"f34a344f-a4cb-442e-b035-9fe2d43ebe80","Type":"ContainerDied","Data":"fdea6e193a43d1732ceec291e74c08b84474d343e4f636c868c4bf7233b26e3c"}
Oct 30 18:33:31.228780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:31.228775    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"
Oct 30 18:33:31.228780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:31.228786    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdea6e193a43d1732ceec291e74c08b84474d343e4f636c868c4bf7233b26e3c"
Oct 30 18:33:34.166461 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:33:34.166423    5746 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-conmon-8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 18:33:34.166862 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:33:34.166479    5746 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 18:33:34.166862 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:33:34.166407    5746 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-conmon-8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf34a344f_a4cb_442e_b035_9fe2d43ebe80.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 18:33:34.166862 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:33:34.166435    5746 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508d4f9d_680b_4e10_9897_4d89dbd6e747.slice/crio-9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e\": RecentStats: unable to find data in memory cache]"
Oct 30 18:33:34.244874 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.244833    5746 generic.go:356] "Generic (PLEG): container finished" podID="508d4f9d-680b-4e10-9897-4d89dbd6e747" containerID="8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457" exitCode=137
Oct 30 18:33:34.245045 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.244927    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/operator-metrics-auth-probe" event={"ID":"508d4f9d-680b-4e10-9897-4d89dbd6e747","Type":"ContainerDied","Data":"8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457"}
Oct 30 18:33:34.292144 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.292117    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:33:34.316613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.316587    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/508d4f9d-680b-4e10-9897-4d89dbd6e747-kube-api-access-9m2lm\" (UniqueName: \"kubernetes.io/projected/508d4f9d-680b-4e10-9897-4d89dbd6e747-kube-api-access-9m2lm\") pod \"508d4f9d-680b-4e10-9897-4d89dbd6e747\" (UID: \"508d4f9d-680b-4e10-9897-4d89dbd6e747\") "
Oct 30 18:33:34.316754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.316657    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/508d4f9d-680b-4e10-9897-4d89dbd6e747-metrics-ca\" (UniqueName: \"kubernetes.io/configmap/508d4f9d-680b-4e10-9897-4d89dbd6e747-metrics-ca\") pod \"508d4f9d-680b-4e10-9897-4d89dbd6e747\" (UID: \"508d4f9d-680b-4e10-9897-4d89dbd6e747\") "
Oct 30 18:33:34.317053 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.317029    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/508d4f9d-680b-4e10-9897-4d89dbd6e747-metrics-ca" pod "508d4f9d-680b-4e10-9897-4d89dbd6e747" (UID: "508d4f9d-680b-4e10-9897-4d89dbd6e747"). InnerVolumeSpecName "metrics-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:33:34.318782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.318762    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/508d4f9d-680b-4e10-9897-4d89dbd6e747-kube-api-access-9m2lm" pod "508d4f9d-680b-4e10-9897-4d89dbd6e747" (UID: "508d4f9d-680b-4e10-9897-4d89dbd6e747"). InnerVolumeSpecName "kube-api-access-9m2lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:33:34.422872 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.422769    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m2lm\" (UniqueName: \"kubernetes.io/projected/508d4f9d-680b-4e10-9897-4d89dbd6e747-kube-api-access-9m2lm\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:34.423087 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:34.422802    5746 reconciler_common.go:299] "Volume detached for volume \"metrics-ca\" (UniqueName: \"kubernetes.io/configmap/508d4f9d-680b-4e10-9897-4d89dbd6e747-metrics-ca\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:35.249712 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:35.249677    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/operator-metrics-auth-probe" event={"ID":"508d4f9d-680b-4e10-9897-4d89dbd6e747","Type":"ContainerDied","Data":"9bf21a4503bdd8de19a5517eb6a70caad5ad6e501f2500be1176959a5fb7c04e"}
Oct 30 18:33:35.250174 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:35.249722    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/operator-metrics-auth-probe"
Oct 30 18:33:35.250174 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:35.249726    5746 scope.go:122] "RemoveContainer" containerID="8124981e2b80a4178abf47224030a01c64a13a190e702461ed656a3d24f30457"
Oct 30 18:33:35.300993 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:35.300963    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/operator-metrics-auth-probe"]
Oct 30 18:33:35.304603 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:35.304582    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/operator-metrics-auth-probe"]
Oct 30 18:33:36.007667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:36.007631    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="508d4f9d-680b-4e10-9897-4d89dbd6e747" path="/var/lib/kubelet/pods/508d4f9d-680b-4e10-9897-4d89dbd6e747/volumes"
Oct 30 18:33:42.790771 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:33:42.790737    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:33:50.352756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.352715    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"]
Oct 30 18:33:50.353195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.353046    5746 cpu_manager.go:414] "containerMap: removing container" podUID="508d4f9d-680b-4e10-9897-4d89dbd6e747" containerName="curl"
Oct 30 18:33:50.353195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.353059    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="508d4f9d-680b-4e10-9897-4d89dbd6e747" containerName="curl"
Oct 30 18:33:50.353195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.353075    5746 cpu_manager.go:414] "containerMap: removing container" podUID="f34a344f-a4cb-442e-b035-9fe2d43ebe80" containerName="curl"
Oct 30 18:33:50.353195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.353080    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f34a344f-a4cb-442e-b035-9fe2d43ebe80" containerName="curl"
Oct 30 18:33:50.353195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.353119    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="508d4f9d-680b-4e10-9897-4d89dbd6e747" containerName="curl"
Oct 30 18:33:50.353195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.353129    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="f34a344f-a4cb-442e-b035-9fe2d43ebe80" containerName="curl"
Oct 30 18:33:50.357262 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.357243    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.387596 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.387570    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"]
Oct 30 18:33:50.455795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.455766    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26nch\" (UniqueName: \"kubernetes.io/projected/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-kube-api-access-26nch\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.455940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.455817    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-ta-internal\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.455940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.455894    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.557306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.557275    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-ta-internal\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.557481 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.557328    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.557481 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.557374    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26nch\" (UniqueName: \"kubernetes.io/projected/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-kube-api-access-26nch\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.558076 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.558044    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-ta-internal\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.560081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.560053    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.581402 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.581376    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26nch\" (UniqueName: \"kubernetes.io/projected/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-kube-api-access-26nch\") pod \"data-science-collector-targetallocator-979d79b87-h2qh5\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.667747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.667683    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:50.963680 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.963593    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"]
Oct 30 18:33:50.970548 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:33:50.970519    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41af1daf_dfb7_4c7c_b861_143aad7ba2a3.slice/crio-e276be7d53236aa5006040c4ecc447ab850e1566fc938ab0660a9a3dd5825e31 WatchSource:0}: Error finding container e276be7d53236aa5006040c4ecc447ab850e1566fc938ab0660a9a3dd5825e31: Status 404 returned error can't find the container with id e276be7d53236aa5006040c4ecc447ab850e1566fc938ab0660a9a3dd5825e31
Oct 30 18:33:50.973037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.973003    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:33:50.973133 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:50.973060    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:33:51.312793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:51.312706    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5" event={"ID":"41af1daf-dfb7-4c7c-b861-143aad7ba2a3","Type":"ContainerStarted","Data":"85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63"}
Oct 30 18:33:51.312793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:51.312748    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5" event={"ID":"41af1daf-dfb7-4c7c-b861-143aad7ba2a3","Type":"ContainerStarted","Data":"e276be7d53236aa5006040c4ecc447ab850e1566fc938ab0660a9a3dd5825e31"}
Oct 30 18:33:51.313059 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:51.312964    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:51.313929 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:51.313888    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5" podUID="41af1daf-dfb7-4c7c-b861-143aad7ba2a3" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.71:8080/readyz\": dial tcp 10.133.0.71:8080: connect: connection refused"
Oct 30 18:33:51.355220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:51.355168    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5" podStartSLOduration=1.355154676 podStartE2EDuration="1.355154676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:51.354613404 +0000 UTC m=+2201.521431543" watchObservedRunningTime="2026-10-30 18:33:51.355154676 +0000 UTC m=+2201.521972830"
Oct 30 18:33:52.317653 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.317626    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:33:52.390392 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.390358    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"]
Oct 30 18:33:52.390801 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.390529    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746" podUID="ddabca8b-da35-4d7d-a04b-8a9af037dd57" containerName="ta-container" containerID="cri-o://02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd" gracePeriod=30
Oct 30 18:33:52.631169 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.631141    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:33:52.774332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.774293    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddabca8b-da35-4d7d-a04b-8a9af037dd57-kube-api-access-vlzm9\" (UniqueName: \"kubernetes.io/projected/ddabca8b-da35-4d7d-a04b-8a9af037dd57-kube-api-access-vlzm9\") pod \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") "
Oct 30 18:33:52.774527 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.774348    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ddabca8b-da35-4d7d-a04b-8a9af037dd57-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ddabca8b-da35-4d7d-a04b-8a9af037dd57-data-science-collector-ta-server-cert\") pod \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") "
Oct 30 18:33:52.774527 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.774391    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddabca8b-da35-4d7d-a04b-8a9af037dd57-ta-internal\" (UniqueName: \"kubernetes.io/configmap/ddabca8b-da35-4d7d-a04b-8a9af037dd57-ta-internal\") pod \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\" (UID: \"ddabca8b-da35-4d7d-a04b-8a9af037dd57\") "
Oct 30 18:33:52.774778 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.774745    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddabca8b-da35-4d7d-a04b-8a9af037dd57-ta-internal" pod "ddabca8b-da35-4d7d-a04b-8a9af037dd57" (UID: "ddabca8b-da35-4d7d-a04b-8a9af037dd57"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:33:52.776647 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.776620    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddabca8b-da35-4d7d-a04b-8a9af037dd57-kube-api-access-vlzm9" pod "ddabca8b-da35-4d7d-a04b-8a9af037dd57" (UID: "ddabca8b-da35-4d7d-a04b-8a9af037dd57"). InnerVolumeSpecName "kube-api-access-vlzm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:33:52.776742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.776651    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddabca8b-da35-4d7d-a04b-8a9af037dd57-data-science-collector-ta-server-cert" pod "ddabca8b-da35-4d7d-a04b-8a9af037dd57" (UID: "ddabca8b-da35-4d7d-a04b-8a9af037dd57"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:33:52.875861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.875788    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlzm9\" (UniqueName: \"kubernetes.io/projected/ddabca8b-da35-4d7d-a04b-8a9af037dd57-kube-api-access-vlzm9\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:52.875861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.875816    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ddabca8b-da35-4d7d-a04b-8a9af037dd57-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:52.875861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:52.875826    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ddabca8b-da35-4d7d-a04b-8a9af037dd57-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:33:53.321083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.320994    5746 generic.go:356] "Generic (PLEG): container finished" podID="ddabca8b-da35-4d7d-a04b-8a9af037dd57" containerID="02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd" exitCode=0
Oct 30 18:33:53.321083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.321042    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"
Oct 30 18:33:53.321318 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.321079    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746" event={"ID":"ddabca8b-da35-4d7d-a04b-8a9af037dd57","Type":"ContainerDied","Data":"02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd"}
Oct 30 18:33:53.321318 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.321120    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746" event={"ID":"ddabca8b-da35-4d7d-a04b-8a9af037dd57","Type":"ContainerDied","Data":"e43fad1a8339f5819e0e308b4267f0ad00cce04bc9e62950f293f8b5dfe3e92e"}
Oct 30 18:33:53.321318 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.321136    5746 scope.go:122] "RemoveContainer" containerID="02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd"
Oct 30 18:33:53.329440 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.329425    5746 scope.go:122] "RemoveContainer" containerID="02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd"
Oct 30 18:33:53.329702 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:33:53.329681    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd\": container with ID starting with 02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd not found: ID does not exist" containerID="02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd"
Oct 30 18:33:53.329765 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.329711    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd"} err="failed to get container status \"02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd\": rpc error: code = NotFound desc = could not find container \"02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd\": container with ID starting with 02e45dd481a6b5245c26cd53be6e4d7ba0d955bd86b0a42efda85995a75e70cd not found: ID does not exist"
Oct 30 18:33:53.371486 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.371462    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"]
Oct 30 18:33:53.383507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.383488    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-26746"]
Oct 30 18:33:53.449864 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.449838    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"]
Oct 30 18:33:53.450200 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.450147    5746 cpu_manager.go:414] "containerMap: removing container" podUID="ddabca8b-da35-4d7d-a04b-8a9af037dd57" containerName="ta-container"
Oct 30 18:33:53.450200 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.450160    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddabca8b-da35-4d7d-a04b-8a9af037dd57" containerName="ta-container"
Oct 30 18:33:53.450274 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.450231    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddabca8b-da35-4d7d-a04b-8a9af037dd57" containerName="ta-container"
Oct 30 18:33:53.454291 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.454277    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"
Oct 30 18:33:53.464343 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.464325    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"
Oct 30 18:33:53.471080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.471062    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"]
Oct 30 18:33:53.758436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:53.758407    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"]
Oct 30 18:33:53.762053 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:33:53.762025    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07fa27e4_5f8e_49ab_b902_b1b5f4304ebc.slice/crio-2780be8f40c814821a7e833b19898ae124defced6b4eb8c805d7831bd8337f3b WatchSource:0}: Error finding container 2780be8f40c814821a7e833b19898ae124defced6b4eb8c805d7831bd8337f3b: Status 404 returned error can't find the container with id 2780be8f40c814821a7e833b19898ae124defced6b4eb8c805d7831bd8337f3b
Oct 30 18:33:54.008587 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:54.008513    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddabca8b-da35-4d7d-a04b-8a9af037dd57" path="/var/lib/kubelet/pods/ddabca8b-da35-4d7d-a04b-8a9af037dd57/volumes"
Oct 30 18:33:54.327646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:54.327572    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68" event={"ID":"07fa27e4-5f8e-49ab-b902-b1b5f4304ebc","Type":"ContainerStarted","Data":"3d521e75c7125e0905862b551f8b650a2b4d4f98cf7ebd5bf08fd8beaa41ecfd"}
Oct 30 18:33:54.327646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:54.327613    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68" event={"ID":"07fa27e4-5f8e-49ab-b902-b1b5f4304ebc","Type":"ContainerStarted","Data":"2780be8f40c814821a7e833b19898ae124defced6b4eb8c805d7831bd8337f3b"}
Oct 30 18:33:54.370205 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:33:54.370153    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68" podStartSLOduration=1.370137825 podStartE2EDuration="1.370137825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:54.369468362 +0000 UTC m=+2204.536286489" watchObservedRunningTime="2026-10-30 18:33:54.370137825 +0000 UTC m=+2204.536955953"
Oct 30 18:34:01.356082 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:01.356052    5746 generic.go:356] "Generic (PLEG): container finished" podID="07fa27e4-5f8e-49ab-b902-b1b5f4304ebc" containerID="3d521e75c7125e0905862b551f8b650a2b4d4f98cf7ebd5bf08fd8beaa41ecfd" exitCode=0
Oct 30 18:34:01.356410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:01.356089    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68" event={"ID":"07fa27e4-5f8e-49ab-b902-b1b5f4304ebc","Type":"ContainerDied","Data":"3d521e75c7125e0905862b551f8b650a2b4d4f98cf7ebd5bf08fd8beaa41ecfd"}
Oct 30 18:34:02.483923 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:02.483880    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"
Oct 30 18:34:03.364528 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.364490    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68" event={"ID":"07fa27e4-5f8e-49ab-b902-b1b5f4304ebc","Type":"ContainerDied","Data":"2780be8f40c814821a7e833b19898ae124defced6b4eb8c805d7831bd8337f3b"}
Oct 30 18:34:03.364528 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.364529    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2780be8f40c814821a7e833b19898ae124defced6b4eb8c805d7831bd8337f3b"
Oct 30 18:34:03.364740 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.364548    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"
Oct 30 18:34:03.623066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.622982    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"]
Oct 30 18:34:03.623531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.623413    5746 cpu_manager.go:414] "containerMap: removing container" podUID="07fa27e4-5f8e-49ab-b902-b1b5f4304ebc" containerName="curl"
Oct 30 18:34:03.623531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.623430    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07fa27e4-5f8e-49ab-b902-b1b5f4304ebc" containerName="curl"
Oct 30 18:34:03.623531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.623512    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="07fa27e4-5f8e-49ab-b902-b1b5f4304ebc" containerName="curl"
Oct 30 18:34:03.626325 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.626303    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"
Oct 30 18:34:03.636860 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.636843    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"
Oct 30 18:34:03.644266 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.644246    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"]
Oct 30 18:34:03.933086 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:03.933014    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"]
Oct 30 18:34:03.936480 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:34:03.936444    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded3fe907_f3bb_4de0_bd25_02e1970585da.slice/crio-c615b4ff51a2ed32ffe7472808477258c058d838fcd59c4d3bb875ac7a1a0368 WatchSource:0}: Error finding container c615b4ff51a2ed32ffe7472808477258c058d838fcd59c4d3bb875ac7a1a0368: Status 404 returned error can't find the container with id c615b4ff51a2ed32ffe7472808477258c058d838fcd59c4d3bb875ac7a1a0368
Oct 30 18:34:04.369595 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:04.369558    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b" event={"ID":"ed3fe907-f3bb-4de0-bd25-02e1970585da","Type":"ContainerStarted","Data":"3a19b91b874c33d22c4a1ecd24bbc8ad71b025b0a33ed0c60c2b525253a88d18"}
Oct 30 18:34:04.369764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:04.369603    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b" event={"ID":"ed3fe907-f3bb-4de0-bd25-02e1970585da","Type":"ContainerStarted","Data":"c615b4ff51a2ed32ffe7472808477258c058d838fcd59c4d3bb875ac7a1a0368"}
Oct 30 18:34:04.408273 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:04.408218    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b" podStartSLOduration=1.408201587 podStartE2EDuration="1.408201587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:04.407972246 +0000 UTC m=+2214.574790384" watchObservedRunningTime="2026-10-30 18:34:04.408201587 +0000 UTC m=+2214.575019746"
Oct 30 18:34:12.400594 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:12.400555    5746 generic.go:356] "Generic (PLEG): container finished" podID="ed3fe907-f3bb-4de0-bd25-02e1970585da" containerID="3a19b91b874c33d22c4a1ecd24bbc8ad71b025b0a33ed0c60c2b525253a88d18" exitCode=0
Oct 30 18:34:12.401089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:12.400608    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b" event={"ID":"ed3fe907-f3bb-4de0-bd25-02e1970585da","Type":"ContainerDied","Data":"3a19b91b874c33d22c4a1ecd24bbc8ad71b025b0a33ed0c60c2b525253a88d18"}
Oct 30 18:34:12.789717 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:34:12.789623    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:34:13.534385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:13.534361    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"
Oct 30 18:34:13.804281 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:13.804202    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"]
Oct 30 18:34:13.804503 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:13.804490    5746 cpu_manager.go:414] "containerMap: removing container" podUID="ed3fe907-f3bb-4de0-bd25-02e1970585da" containerName="curl"
Oct 30 18:34:13.804544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:13.804504    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed3fe907-f3bb-4de0-bd25-02e1970585da" containerName="curl"
Oct 30 18:34:13.804579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:13.804548    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed3fe907-f3bb-4de0-bd25-02e1970585da" containerName="curl"
Oct 30 18:34:13.807202 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:13.807187    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"
Oct 30 18:34:13.817582 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:13.817551    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"
Oct 30 18:34:13.824966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:13.824941    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"]
Oct 30 18:34:14.112594 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:14.112567    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"]
Oct 30 18:34:14.115402 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:34:14.115379    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode113a893_9fb9_43c8_a1d5_ec05590e85a8.slice/crio-0ff9dd9f977967ab6e544edc3964032562b7cf6247a7a490ca6bc839c3f522bc WatchSource:0}: Error finding container 0ff9dd9f977967ab6e544edc3964032562b7cf6247a7a490ca6bc839c3f522bc: Status 404 returned error can't find the container with id 0ff9dd9f977967ab6e544edc3964032562b7cf6247a7a490ca6bc839c3f522bc
Oct 30 18:34:14.411268 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:14.411190    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"
Oct 30 18:34:14.411423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:14.411187    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b" event={"ID":"ed3fe907-f3bb-4de0-bd25-02e1970585da","Type":"ContainerDied","Data":"c615b4ff51a2ed32ffe7472808477258c058d838fcd59c4d3bb875ac7a1a0368"}
Oct 30 18:34:14.411423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:14.411320    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c615b4ff51a2ed32ffe7472808477258c058d838fcd59c4d3bb875ac7a1a0368"
Oct 30 18:34:14.412675 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:14.412649    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce" event={"ID":"e113a893-9fb9-43c8-a1d5-ec05590e85a8","Type":"ContainerStarted","Data":"a60349a53a92b0bb08caa466f7a24c8aa2f2ec9aa8c24df061ec0572ccfd77fe"}
Oct 30 18:34:14.412823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:14.412685    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce" event={"ID":"e113a893-9fb9-43c8-a1d5-ec05590e85a8","Type":"ContainerStarted","Data":"0ff9dd9f977967ab6e544edc3964032562b7cf6247a7a490ca6bc839c3f522bc"}
Oct 30 18:34:14.451883 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:14.451833    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce" podStartSLOduration=1.4518166780000001 podStartE2EDuration="1.451816678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:14.451192257 +0000 UTC m=+2224.618010379" watchObservedRunningTime="2026-10-30 18:34:14.451816678 +0000 UTC m=+2224.618634805"
Oct 30 18:34:22.444037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:22.444001    5746 generic.go:356] "Generic (PLEG): container finished" podID="e113a893-9fb9-43c8-a1d5-ec05590e85a8" containerID="a60349a53a92b0bb08caa466f7a24c8aa2f2ec9aa8c24df061ec0572ccfd77fe" exitCode=0
Oct 30 18:34:22.444461 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:22.444075    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce" event={"ID":"e113a893-9fb9-43c8-a1d5-ec05590e85a8","Type":"ContainerDied","Data":"a60349a53a92b0bb08caa466f7a24c8aa2f2ec9aa8c24df061ec0572ccfd77fe"}
Oct 30 18:34:23.580109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:23.580085    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"
Oct 30 18:34:23.976207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:23.976174    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"]
Oct 30 18:34:23.976471 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:23.976458    5746 cpu_manager.go:414] "containerMap: removing container" podUID="e113a893-9fb9-43c8-a1d5-ec05590e85a8" containerName="curl"
Oct 30 18:34:23.976515 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:23.976473    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e113a893-9fb9-43c8-a1d5-ec05590e85a8" containerName="curl"
Oct 30 18:34:23.976551 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:23.976531    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="e113a893-9fb9-43c8-a1d5-ec05590e85a8" containerName="curl"
Oct 30 18:34:23.979380 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:23.979360    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"
Oct 30 18:34:23.989748 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:23.989726    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"
Oct 30 18:34:23.997024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:23.996999    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"]
Oct 30 18:34:24.286865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:24.286786    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"]
Oct 30 18:34:24.290567 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:34:24.290540    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06ea13c7_83f8_43e5_8a1f_56c17cf29479.slice/crio-5af23522dd00f14e132b0d47eb28fdeeaed57e1e57ce09b1f72a596178d507db WatchSource:0}: Error finding container 5af23522dd00f14e132b0d47eb28fdeeaed57e1e57ce09b1f72a596178d507db: Status 404 returned error can't find the container with id 5af23522dd00f14e132b0d47eb28fdeeaed57e1e57ce09b1f72a596178d507db
Oct 30 18:34:24.452947 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:24.452894    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"
Oct 30 18:34:24.453150 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:24.452891    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce" event={"ID":"e113a893-9fb9-43c8-a1d5-ec05590e85a8","Type":"ContainerDied","Data":"0ff9dd9f977967ab6e544edc3964032562b7cf6247a7a490ca6bc839c3f522bc"}
Oct 30 18:34:24.453150 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:24.453057    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ff9dd9f977967ab6e544edc3964032562b7cf6247a7a490ca6bc839c3f522bc"
Oct 30 18:34:24.454419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:24.454395    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a" event={"ID":"06ea13c7-83f8-43e5-8a1f-56c17cf29479","Type":"ContainerStarted","Data":"ddb68816ca5d5a50de1fb2ccae9c8548b1634fbd0735aac1e77b3b4604a2178c"}
Oct 30 18:34:24.454532 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:24.454430    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a" event={"ID":"06ea13c7-83f8-43e5-8a1f-56c17cf29479","Type":"ContainerStarted","Data":"5af23522dd00f14e132b0d47eb28fdeeaed57e1e57ce09b1f72a596178d507db"}
Oct 30 18:34:24.494809 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:24.494756    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a" podStartSLOduration=1.494743861 podStartE2EDuration="1.494743861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:24.494010298 +0000 UTC m=+2234.660828427" watchObservedRunningTime="2026-10-30 18:34:24.494743861 +0000 UTC m=+2234.661561984"
Oct 30 18:34:32.494257 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:32.494223    5746 generic.go:356] "Generic (PLEG): container finished" podID="06ea13c7-83f8-43e5-8a1f-56c17cf29479" containerID="ddb68816ca5d5a50de1fb2ccae9c8548b1634fbd0735aac1e77b3b4604a2178c" exitCode=0
Oct 30 18:34:32.494696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:32.494263    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a" event={"ID":"06ea13c7-83f8-43e5-8a1f-56c17cf29479","Type":"ContainerDied","Data":"ddb68816ca5d5a50de1fb2ccae9c8548b1634fbd0735aac1e77b3b4604a2178c"}
Oct 30 18:34:33.625242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:33.625217    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"
Oct 30 18:34:34.147656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.147617    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"]
Oct 30 18:34:34.147955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.147942    5746 cpu_manager.go:414] "containerMap: removing container" podUID="06ea13c7-83f8-43e5-8a1f-56c17cf29479" containerName="curl"
Oct 30 18:34:34.148010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.147957    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06ea13c7-83f8-43e5-8a1f-56c17cf29479" containerName="curl"
Oct 30 18:34:34.148010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.148007    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="06ea13c7-83f8-43e5-8a1f-56c17cf29479" containerName="curl"
Oct 30 18:34:34.150790 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.150772    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"
Oct 30 18:34:34.161115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.161097    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"
Oct 30 18:34:34.174667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.174283    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"]
Oct 30 18:34:34.463208 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.463178    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"]
Oct 30 18:34:34.466707 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:34:34.466677    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfb38fae_5220_409f_9dde_ada0d16ec40d.slice/crio-4cf9f93eb438ccdf658d7b9f3ebffd4227aa7789eeef0f393ad55b1cf53bf261 WatchSource:0}: Error finding container 4cf9f93eb438ccdf658d7b9f3ebffd4227aa7789eeef0f393ad55b1cf53bf261: Status 404 returned error can't find the container with id 4cf9f93eb438ccdf658d7b9f3ebffd4227aa7789eeef0f393ad55b1cf53bf261
Oct 30 18:34:34.503872 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.503843    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610" event={"ID":"cfb38fae-5220-409f-9dde-ada0d16ec40d","Type":"ContainerStarted","Data":"4cf9f93eb438ccdf658d7b9f3ebffd4227aa7789eeef0f393ad55b1cf53bf261"}
Oct 30 18:34:34.505434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.505405    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a" event={"ID":"06ea13c7-83f8-43e5-8a1f-56c17cf29479","Type":"ContainerDied","Data":"5af23522dd00f14e132b0d47eb28fdeeaed57e1e57ce09b1f72a596178d507db"}
Oct 30 18:34:34.505537 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.505436    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"
Oct 30 18:34:34.505537 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:34.505446    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5af23522dd00f14e132b0d47eb28fdeeaed57e1e57ce09b1f72a596178d507db"
Oct 30 18:34:35.511275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:35.511239    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610" event={"ID":"cfb38fae-5220-409f-9dde-ada0d16ec40d","Type":"ContainerStarted","Data":"cdd602568d8daa1ecb50f7d0172074b3ea0159c65d5e8539b8ec50eede7cbae1"}
Oct 30 18:34:35.549361 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:35.549312    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610" podStartSLOduration=1.5492991520000001 podStartE2EDuration="1.549299152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:35.548900056 +0000 UTC m=+2245.715718184" watchObservedRunningTime="2026-10-30 18:34:35.549299152 +0000 UTC m=+2245.716117273"
Oct 30 18:34:42.539657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:42.539626    5746 generic.go:356] "Generic (PLEG): container finished" podID="cfb38fae-5220-409f-9dde-ada0d16ec40d" containerID="cdd602568d8daa1ecb50f7d0172074b3ea0159c65d5e8539b8ec50eede7cbae1" exitCode=0
Oct 30 18:34:42.540165 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:42.539668    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610" event={"ID":"cfb38fae-5220-409f-9dde-ada0d16ec40d","Type":"ContainerDied","Data":"cdd602568d8daa1ecb50f7d0172074b3ea0159c65d5e8539b8ec50eede7cbae1"}
Oct 30 18:34:42.790271 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:34:42.790190    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:34:43.677442 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:43.677419    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"
Oct 30 18:34:44.321456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.321377    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"]
Oct 30 18:34:44.321689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.321677    5746 cpu_manager.go:414] "containerMap: removing container" podUID="cfb38fae-5220-409f-9dde-ada0d16ec40d" containerName="curl"
Oct 30 18:34:44.321732 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.321690    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfb38fae-5220-409f-9dde-ada0d16ec40d" containerName="curl"
Oct 30 18:34:44.321771 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.321737    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfb38fae-5220-409f-9dde-ada0d16ec40d" containerName="curl"
Oct 30 18:34:44.324448 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.324430    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"
Oct 30 18:34:44.335105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.335081    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"
Oct 30 18:34:44.342202 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.342182    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"]
Oct 30 18:34:44.548807 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.548770    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"
Oct 30 18:34:44.548807 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.548786    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610" event={"ID":"cfb38fae-5220-409f-9dde-ada0d16ec40d","Type":"ContainerDied","Data":"4cf9f93eb438ccdf658d7b9f3ebffd4227aa7789eeef0f393ad55b1cf53bf261"}
Oct 30 18:34:44.549041 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.548821    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cf9f93eb438ccdf658d7b9f3ebffd4227aa7789eeef0f393ad55b1cf53bf261"
Oct 30 18:34:44.636037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:44.636008    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"]
Oct 30 18:34:44.639566 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:34:44.639539    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4306185_d4af_4bd9_98da_40674725773c.slice/crio-4c1e13a62e5a3d7a484313b56e6040d29dd3b402e847b3d21f0adc14b482f72a WatchSource:0}: Error finding container 4c1e13a62e5a3d7a484313b56e6040d29dd3b402e847b3d21f0adc14b482f72a: Status 404 returned error can't find the container with id 4c1e13a62e5a3d7a484313b56e6040d29dd3b402e847b3d21f0adc14b482f72a
Oct 30 18:34:45.554550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:45.554512    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e" event={"ID":"f4306185-d4af-4bd9-98da-40674725773c","Type":"ContainerStarted","Data":"460915081cbb634242c0e8e60cd3f7574901313b2d676cddd6440a62905ccc58"}
Oct 30 18:34:45.555004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:45.554562    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e" event={"ID":"f4306185-d4af-4bd9-98da-40674725773c","Type":"ContainerStarted","Data":"4c1e13a62e5a3d7a484313b56e6040d29dd3b402e847b3d21f0adc14b482f72a"}
Oct 30 18:34:52.591090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:52.590995    5746 generic.go:356] "Generic (PLEG): container finished" podID="f4306185-d4af-4bd9-98da-40674725773c" containerID="460915081cbb634242c0e8e60cd3f7574901313b2d676cddd6440a62905ccc58" exitCode=0
Oct 30 18:34:52.591090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:52.591064    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e" event={"ID":"f4306185-d4af-4bd9-98da-40674725773c","Type":"ContainerDied","Data":"460915081cbb634242c0e8e60cd3f7574901313b2d676cddd6440a62905ccc58"}
Oct 30 18:34:53.718107 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:53.718079    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"
Oct 30 18:34:54.496306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.496269    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"]
Oct 30 18:34:54.496619 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.496602    5746 cpu_manager.go:414] "containerMap: removing container" podUID="f4306185-d4af-4bd9-98da-40674725773c" containerName="curl"
Oct 30 18:34:54.496696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.496621    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4306185-d4af-4bd9-98da-40674725773c" containerName="curl"
Oct 30 18:34:54.496744 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.496719    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4306185-d4af-4bd9-98da-40674725773c" containerName="curl"
Oct 30 18:34:54.499412 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.499392    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"
Oct 30 18:34:54.509475 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.509456    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"
Oct 30 18:34:54.517102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.517081    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"]
Oct 30 18:34:54.599866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.599838    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e" event={"ID":"f4306185-d4af-4bd9-98da-40674725773c","Type":"ContainerDied","Data":"4c1e13a62e5a3d7a484313b56e6040d29dd3b402e847b3d21f0adc14b482f72a"}
Oct 30 18:34:54.600029 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.599869    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"
Oct 30 18:34:54.600029 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.599875    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c1e13a62e5a3d7a484313b56e6040d29dd3b402e847b3d21f0adc14b482f72a"
Oct 30 18:34:54.804285 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:54.804198    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"]
Oct 30 18:34:54.807469 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:34:54.807438    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21bbd240_586f_428f_a22b_d525c7f035fb.slice/crio-81c53c558a287ee1de6eff00ac6b6c49ee3c792fb6bf11d0c6e50f5cb7c84dfe WatchSource:0}: Error finding container 81c53c558a287ee1de6eff00ac6b6c49ee3c792fb6bf11d0c6e50f5cb7c84dfe: Status 404 returned error can't find the container with id 81c53c558a287ee1de6eff00ac6b6c49ee3c792fb6bf11d0c6e50f5cb7c84dfe
Oct 30 18:34:55.605527 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:55.605490    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6" event={"ID":"21bbd240-586f-428f-a22b-d525c7f035fb","Type":"ContainerStarted","Data":"7d3aebed6696915b325ffe2152247ab6f7f1ea5cc218ec797d31c37c95bd2729"}
Oct 30 18:34:55.605707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:55.605538    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6" event={"ID":"21bbd240-586f-428f-a22b-d525c7f035fb","Type":"ContainerStarted","Data":"81c53c558a287ee1de6eff00ac6b6c49ee3c792fb6bf11d0c6e50f5cb7c84dfe"}
Oct 30 18:34:55.644709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:34:55.644660    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6" podStartSLOduration=1.644647123 podStartE2EDuration="1.644647123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:55.643618033 +0000 UTC m=+2265.810436170" watchObservedRunningTime="2026-10-30 18:34:55.644647123 +0000 UTC m=+2265.811465250"
Oct 30 18:35:02.634764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:02.634727    5746 generic.go:356] "Generic (PLEG): container finished" podID="21bbd240-586f-428f-a22b-d525c7f035fb" containerID="7d3aebed6696915b325ffe2152247ab6f7f1ea5cc218ec797d31c37c95bd2729" exitCode=0
Oct 30 18:35:02.635175 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:02.634806    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6" event={"ID":"21bbd240-586f-428f-a22b-d525c7f035fb","Type":"ContainerDied","Data":"7d3aebed6696915b325ffe2152247ab6f7f1ea5cc218ec797d31c37c95bd2729"}
Oct 30 18:35:03.766517 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:03.766495    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"
Oct 30 18:35:04.644144 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.644111    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6" event={"ID":"21bbd240-586f-428f-a22b-d525c7f035fb","Type":"ContainerDied","Data":"81c53c558a287ee1de6eff00ac6b6c49ee3c792fb6bf11d0c6e50f5cb7c84dfe"}
Oct 30 18:35:04.644144 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.644142    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"
Oct 30 18:35:04.644352 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.644156    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81c53c558a287ee1de6eff00ac6b6c49ee3c792fb6bf11d0c6e50f5cb7c84dfe"
Oct 30 18:35:04.668118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.668083    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"]
Oct 30 18:35:04.668424 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.668408    5746 cpu_manager.go:414] "containerMap: removing container" podUID="21bbd240-586f-428f-a22b-d525c7f035fb" containerName="curl"
Oct 30 18:35:04.668508 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.668427    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21bbd240-586f-428f-a22b-d525c7f035fb" containerName="curl"
Oct 30 18:35:04.668564 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.668520    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="21bbd240-586f-428f-a22b-d525c7f035fb" containerName="curl"
Oct 30 18:35:04.671361 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.671341    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"
Oct 30 18:35:04.681589 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.681560    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"
Oct 30 18:35:04.688999 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.688973    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"]
Oct 30 18:35:04.980469 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:04.980398    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"]
Oct 30 18:35:04.983962 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:35:04.983934    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda40bb14a_083b_4efb_b95a_7d18fa10881f.slice/crio-22f6b3ce824a4bcbaaffb545a8aaa0ac3893aebb1728b903494d302b0bb4afec WatchSource:0}: Error finding container 22f6b3ce824a4bcbaaffb545a8aaa0ac3893aebb1728b903494d302b0bb4afec: Status 404 returned error can't find the container with id 22f6b3ce824a4bcbaaffb545a8aaa0ac3893aebb1728b903494d302b0bb4afec
Oct 30 18:35:05.649795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:05.649747    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb" event={"ID":"a40bb14a-083b-4efb-b95a-7d18fa10881f","Type":"ContainerStarted","Data":"56b7cba45a6af504ab179bf491a552356cf1e125fc5f66edee41db97d6d7d0f5"}
Oct 30 18:35:05.650020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:05.649802    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb" event={"ID":"a40bb14a-083b-4efb-b95a-7d18fa10881f","Type":"ContainerStarted","Data":"22f6b3ce824a4bcbaaffb545a8aaa0ac3893aebb1728b903494d302b0bb4afec"}
Oct 30 18:35:05.687878 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:05.687821    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb" podStartSLOduration=1.687809213 podStartE2EDuration="1.687809213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:05.68726673 +0000 UTC m=+2275.854084872" watchObservedRunningTime="2026-10-30 18:35:05.687809213 +0000 UTC m=+2275.854627341"
Oct 30 18:35:12.680567 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:12.680533    5746 generic.go:356] "Generic (PLEG): container finished" podID="a40bb14a-083b-4efb-b95a-7d18fa10881f" containerID="56b7cba45a6af504ab179bf491a552356cf1e125fc5f66edee41db97d6d7d0f5" exitCode=0
Oct 30 18:35:12.680961 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:12.680600    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb" event={"ID":"a40bb14a-083b-4efb-b95a-7d18fa10881f","Type":"ContainerDied","Data":"56b7cba45a6af504ab179bf491a552356cf1e125fc5f66edee41db97d6d7d0f5"}
Oct 30 18:35:12.789569 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:35:12.789531    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:35:13.821560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:13.821534    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"
Oct 30 18:35:14.689515 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.689484    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"
Oct 30 18:35:14.689515 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.689492    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb" event={"ID":"a40bb14a-083b-4efb-b95a-7d18fa10881f","Type":"ContainerDied","Data":"22f6b3ce824a4bcbaaffb545a8aaa0ac3893aebb1728b903494d302b0bb4afec"}
Oct 30 18:35:14.689755 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.689526    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22f6b3ce824a4bcbaaffb545a8aaa0ac3893aebb1728b903494d302b0bb4afec"
Oct 30 18:35:14.840866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.840831    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"]
Oct 30 18:35:14.841365 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.841255    5746 cpu_manager.go:414] "containerMap: removing container" podUID="a40bb14a-083b-4efb-b95a-7d18fa10881f" containerName="curl"
Oct 30 18:35:14.841365 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.841273    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a40bb14a-083b-4efb-b95a-7d18fa10881f" containerName="curl"
Oct 30 18:35:14.841365 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.841365    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="a40bb14a-083b-4efb-b95a-7d18fa10881f" containerName="curl"
Oct 30 18:35:14.844216 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.844195    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"
Oct 30 18:35:14.854384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.854367    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"
Oct 30 18:35:14.861447 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:14.861361    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"]
Oct 30 18:35:15.150024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:15.149996    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"]
Oct 30 18:35:15.153655 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:35:15.153627    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c0c7390_ad9b_4fa7_b26d_dc869bc373fd.slice/crio-1da77942b3913f8876b2ebfa7d16ccbff193b0f644fe63ad2cf7712c34df2499 WatchSource:0}: Error finding container 1da77942b3913f8876b2ebfa7d16ccbff193b0f644fe63ad2cf7712c34df2499: Status 404 returned error can't find the container with id 1da77942b3913f8876b2ebfa7d16ccbff193b0f644fe63ad2cf7712c34df2499
Oct 30 18:35:15.695586 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:15.695553    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c" event={"ID":"7c0c7390-ad9b-4fa7-b26d-dc869bc373fd","Type":"ContainerStarted","Data":"a08ce1fcff8244700fccebe139ea641d7ced69374721836bf05835cec941f27a"}
Oct 30 18:35:15.695754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:15.695594    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c" event={"ID":"7c0c7390-ad9b-4fa7-b26d-dc869bc373fd","Type":"ContainerStarted","Data":"1da77942b3913f8876b2ebfa7d16ccbff193b0f644fe63ad2cf7712c34df2499"}
Oct 30 18:35:15.733707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:15.733655    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c" podStartSLOduration=1.733641273 podStartE2EDuration="1.733641273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:15.732860512 +0000 UTC m=+2285.899678659" watchObservedRunningTime="2026-10-30 18:35:15.733641273 +0000 UTC m=+2285.900459400"
Oct 30 18:35:23.729886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:23.729852    5746 generic.go:356] "Generic (PLEG): container finished" podID="7c0c7390-ad9b-4fa7-b26d-dc869bc373fd" containerID="a08ce1fcff8244700fccebe139ea641d7ced69374721836bf05835cec941f27a" exitCode=0
Oct 30 18:35:23.730389 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:23.729899    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c" event={"ID":"7c0c7390-ad9b-4fa7-b26d-dc869bc373fd","Type":"ContainerDied","Data":"a08ce1fcff8244700fccebe139ea641d7ced69374721836bf05835cec941f27a"}
Oct 30 18:35:24.862531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:24.862504    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"
Oct 30 18:35:25.014628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.014537    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"]
Oct 30 18:35:25.014945 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.014929    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7c0c7390-ad9b-4fa7-b26d-dc869bc373fd" containerName="curl"
Oct 30 18:35:25.015026 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.014948    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c0c7390-ad9b-4fa7-b26d-dc869bc373fd" containerName="curl"
Oct 30 18:35:25.015091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.015045    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c0c7390-ad9b-4fa7-b26d-dc869bc373fd" containerName="curl"
Oct 30 18:35:25.017925 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.017880    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"
Oct 30 18:35:25.028250 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.028229    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"
Oct 30 18:35:25.035259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.035236    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"]
Oct 30 18:35:25.325151 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.325074    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"]
Oct 30 18:35:25.328617 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:35:25.328589    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9019d2a3_dd51_46bf_adb3_f4180a465efe.slice/crio-a48c559563af5c2dfea9e00c616b04e3e8ccefe6f31c48bd3d420fca0b5abbdf WatchSource:0}: Error finding container a48c559563af5c2dfea9e00c616b04e3e8ccefe6f31c48bd3d420fca0b5abbdf: Status 404 returned error can't find the container with id a48c559563af5c2dfea9e00c616b04e3e8ccefe6f31c48bd3d420fca0b5abbdf
Oct 30 18:35:25.740347 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.740320    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"
Oct 30 18:35:25.740536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.740323    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c" event={"ID":"7c0c7390-ad9b-4fa7-b26d-dc869bc373fd","Type":"ContainerDied","Data":"1da77942b3913f8876b2ebfa7d16ccbff193b0f644fe63ad2cf7712c34df2499"}
Oct 30 18:35:25.740536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.740454    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1da77942b3913f8876b2ebfa7d16ccbff193b0f644fe63ad2cf7712c34df2499"
Oct 30 18:35:25.741872 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.741849    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c" event={"ID":"9019d2a3-dd51-46bf-adb3-f4180a465efe","Type":"ContainerStarted","Data":"6d4826b0f1b669f9d0e4562ec7d6d2d23406d468f798c7cc0d961967c0935359"}
Oct 30 18:35:25.742019 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.741880    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c" event={"ID":"9019d2a3-dd51-46bf-adb3-f4180a465efe","Type":"ContainerStarted","Data":"a48c559563af5c2dfea9e00c616b04e3e8ccefe6f31c48bd3d420fca0b5abbdf"}
Oct 30 18:35:25.780259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:25.780205    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c" podStartSLOduration=1.780189187 podStartE2EDuration="1.780189187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:25.780091265 +0000 UTC m=+2295.946909393" watchObservedRunningTime="2026-10-30 18:35:25.780189187 +0000 UTC m=+2295.947007315"
Oct 30 18:35:33.776428 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:33.776393    5746 generic.go:356] "Generic (PLEG): container finished" podID="9019d2a3-dd51-46bf-adb3-f4180a465efe" containerID="6d4826b0f1b669f9d0e4562ec7d6d2d23406d468f798c7cc0d961967c0935359" exitCode=0
Oct 30 18:35:33.776950 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:33.776470    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c" event={"ID":"9019d2a3-dd51-46bf-adb3-f4180a465efe","Type":"ContainerDied","Data":"6d4826b0f1b669f9d0e4562ec7d6d2d23406d468f798c7cc0d961967c0935359"}
Oct 30 18:35:34.905861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:34.905838    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"
Oct 30 18:35:35.190263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.190231    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"]
Oct 30 18:35:35.190522 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.190510    5746 cpu_manager.go:414] "containerMap: removing container" podUID="9019d2a3-dd51-46bf-adb3-f4180a465efe" containerName="curl"
Oct 30 18:35:35.190573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.190523    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9019d2a3-dd51-46bf-adb3-f4180a465efe" containerName="curl"
Oct 30 18:35:35.190608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.190586    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="9019d2a3-dd51-46bf-adb3-f4180a465efe" containerName="curl"
Oct 30 18:35:35.193348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.193331    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"
Oct 30 18:35:35.203778 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.203760    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"
Oct 30 18:35:35.211741 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.211719    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"]
Oct 30 18:35:35.498138 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.498056    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"]
Oct 30 18:35:35.501093 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:35:35.501064    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70897f9d_8556_4762_af73_21bb38f543f9.slice/crio-d41a78a40b8edcc4549736ffcfc55bfe01dfca28440bb9b3399ccf2ded868807 WatchSource:0}: Error finding container d41a78a40b8edcc4549736ffcfc55bfe01dfca28440bb9b3399ccf2ded868807: Status 404 returned error can't find the container with id d41a78a40b8edcc4549736ffcfc55bfe01dfca28440bb9b3399ccf2ded868807
Oct 30 18:35:35.786090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.785998    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4" event={"ID":"70897f9d-8556-4762-af73-21bb38f543f9","Type":"ContainerStarted","Data":"b4dc892a82a4dfd7409d385a13156f118502d7e6eb3c9b9e46fa26fc88fbc1dc"}
Oct 30 18:35:35.786090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.786044    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4" event={"ID":"70897f9d-8556-4762-af73-21bb38f543f9","Type":"ContainerStarted","Data":"d41a78a40b8edcc4549736ffcfc55bfe01dfca28440bb9b3399ccf2ded868807"}
Oct 30 18:35:35.787253 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.787230    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"
Oct 30 18:35:35.787253 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.787260    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c" event={"ID":"9019d2a3-dd51-46bf-adb3-f4180a465efe","Type":"ContainerDied","Data":"a48c559563af5c2dfea9e00c616b04e3e8ccefe6f31c48bd3d420fca0b5abbdf"}
Oct 30 18:35:35.787462 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.787286    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a48c559563af5c2dfea9e00c616b04e3e8ccefe6f31c48bd3d420fca0b5abbdf"
Oct 30 18:35:35.824646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:35.824595    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4" podStartSLOduration=0.82458072 podStartE2EDuration="824.58072ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:35.824157957 +0000 UTC m=+2305.990976085" watchObservedRunningTime="2026-10-30 18:35:35.82458072 +0000 UTC m=+2305.991398844"
Oct 30 18:35:42.790296 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:35:42.790261    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:35:43.823106 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:43.823073    5746 generic.go:356] "Generic (PLEG): container finished" podID="70897f9d-8556-4762-af73-21bb38f543f9" containerID="b4dc892a82a4dfd7409d385a13156f118502d7e6eb3c9b9e46fa26fc88fbc1dc" exitCode=0
Oct 30 18:35:43.823106 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:43.823109    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4" event={"ID":"70897f9d-8556-4762-af73-21bb38f543f9","Type":"ContainerDied","Data":"b4dc892a82a4dfd7409d385a13156f118502d7e6eb3c9b9e46fa26fc88fbc1dc"}
Oct 30 18:35:44.958070 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:44.958045    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"
Oct 30 18:35:45.378495 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.378464    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"]
Oct 30 18:35:45.378791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.378778    5746 cpu_manager.go:414] "containerMap: removing container" podUID="70897f9d-8556-4762-af73-21bb38f543f9" containerName="curl"
Oct 30 18:35:45.378835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.378793    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70897f9d-8556-4762-af73-21bb38f543f9" containerName="curl"
Oct 30 18:35:45.378869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.378843    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="70897f9d-8556-4762-af73-21bb38f543f9" containerName="curl"
Oct 30 18:35:45.381726 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.381710    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"
Oct 30 18:35:45.399346 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.399320    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"]
Oct 30 18:35:45.410350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.410330    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-network-policy-18e3625bcbbc44ff\"/\"default-dockercfg-w486b\""
Oct 30 18:35:45.412974 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.412958    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"
Oct 30 18:35:45.706456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.706378    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"]
Oct 30 18:35:45.709669 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:35:45.709643    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5be0df53_5fd1_481e_9d00_5e8c022ee521.slice/crio-275969bc0aff1ef59ab38f0fc5e79ea82edd1cd4f315133aa33e0afd30729ef1 WatchSource:0}: Error finding container 275969bc0aff1ef59ab38f0fc5e79ea82edd1cd4f315133aa33e0afd30729ef1: Status 404 returned error can't find the container with id 275969bc0aff1ef59ab38f0fc5e79ea82edd1cd4f315133aa33e0afd30729ef1
Oct 30 18:35:45.837767 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.837741    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"
Oct 30 18:35:45.837953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.837740    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4" event={"ID":"70897f9d-8556-4762-af73-21bb38f543f9","Type":"ContainerDied","Data":"d41a78a40b8edcc4549736ffcfc55bfe01dfca28440bb9b3399ccf2ded868807"}
Oct 30 18:35:45.837953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.837861    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d41a78a40b8edcc4549736ffcfc55bfe01dfca28440bb9b3399ccf2ded868807"
Oct 30 18:35:45.840244 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.840212    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba" event={"ID":"5be0df53-5fd1-481e-9d00-5e8c022ee521","Type":"ContainerStarted","Data":"f5b47f4c212adcb522b78902cab7cfa47977ea62ffbc830a81ac000c0edaa1ec"}
Oct 30 18:35:45.840442 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:45.840412    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba" event={"ID":"5be0df53-5fd1-481e-9d00-5e8c022ee521","Type":"ContainerStarted","Data":"275969bc0aff1ef59ab38f0fc5e79ea82edd1cd4f315133aa33e0afd30729ef1"}
Oct 30 18:35:46.845731 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:46.845693    5746 generic.go:356] "Generic (PLEG): container finished" podID="5be0df53-5fd1-481e-9d00-5e8c022ee521" containerID="f5b47f4c212adcb522b78902cab7cfa47977ea62ffbc830a81ac000c0edaa1ec" exitCode=0
Oct 30 18:35:46.846163 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:46.845751    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba" event={"ID":"5be0df53-5fd1-481e-9d00-5e8c022ee521","Type":"ContainerDied","Data":"f5b47f4c212adcb522b78902cab7cfa47977ea62ffbc830a81ac000c0edaa1ec"}
Oct 30 18:35:46.963525 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:46.963502    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"
Oct 30 18:35:47.479754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.479718    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"]
Oct 30 18:35:47.480030 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.480017    5746 cpu_manager.go:414] "containerMap: removing container" podUID="5be0df53-5fd1-481e-9d00-5e8c022ee521" containerName="curl"
Oct 30 18:35:47.480093 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.480031    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5be0df53-5fd1-481e-9d00-5e8c022ee521" containerName="curl"
Oct 30 18:35:47.480093 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.480089    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="5be0df53-5fd1-481e-9d00-5e8c022ee521" containerName="curl"
Oct 30 18:35:47.483033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.483013    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"
Oct 30 18:35:47.493616 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.493594    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"
Oct 30 18:35:47.500798 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.500773    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"]
Oct 30 18:35:47.790431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.790360    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"]
Oct 30 18:35:47.793158 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:35:47.793120    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc68792d_6dde_4398_a24d_eee8c143d22a.slice/crio-a06b4ec4b52908f1c225ba7248bc87e31be340b234fdbaabde18a84a34ef8975 WatchSource:0}: Error finding container a06b4ec4b52908f1c225ba7248bc87e31be340b234fdbaabde18a84a34ef8975: Status 404 returned error can't find the container with id a06b4ec4b52908f1c225ba7248bc87e31be340b234fdbaabde18a84a34ef8975
Oct 30 18:35:47.852172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.852143    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"
Oct 30 18:35:47.852491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.852175    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba" event={"ID":"5be0df53-5fd1-481e-9d00-5e8c022ee521","Type":"ContainerDied","Data":"275969bc0aff1ef59ab38f0fc5e79ea82edd1cd4f315133aa33e0afd30729ef1"}
Oct 30 18:35:47.852491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.852219    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="275969bc0aff1ef59ab38f0fc5e79ea82edd1cd4f315133aa33e0afd30729ef1"
Oct 30 18:35:47.853838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.853814    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02" event={"ID":"fc68792d-6dde-4398-a24d-eee8c143d22a","Type":"ContainerStarted","Data":"808e0d625c6a8a52648b1dfbe6050b89c048aa8e24bd9e2e1d44d73b50b1ceac"}
Oct 30 18:35:47.853961 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:47.853852    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02" event={"ID":"fc68792d-6dde-4398-a24d-eee8c143d22a","Type":"ContainerStarted","Data":"a06b4ec4b52908f1c225ba7248bc87e31be340b234fdbaabde18a84a34ef8975"}
Oct 30 18:35:48.859604 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:48.859571    5746 generic.go:356] "Generic (PLEG): container finished" podID="fc68792d-6dde-4398-a24d-eee8c143d22a" containerID="808e0d625c6a8a52648b1dfbe6050b89c048aa8e24bd9e2e1d44d73b50b1ceac" exitCode=0
Oct 30 18:35:48.860029 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:48.859659    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02" event={"ID":"fc68792d-6dde-4398-a24d-eee8c143d22a","Type":"ContainerDied","Data":"808e0d625c6a8a52648b1dfbe6050b89c048aa8e24bd9e2e1d44d73b50b1ceac"}
Oct 30 18:35:49.994748 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:49.994723    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"
Oct 30 18:35:50.871139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:50.871103    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02" event={"ID":"fc68792d-6dde-4398-a24d-eee8c143d22a","Type":"ContainerDied","Data":"a06b4ec4b52908f1c225ba7248bc87e31be340b234fdbaabde18a84a34ef8975"}
Oct 30 18:35:50.871139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:50.871137    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"
Oct 30 18:35:50.871348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:50.871147    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a06b4ec4b52908f1c225ba7248bc87e31be340b234fdbaabde18a84a34ef8975"
Oct 30 18:35:51.610036 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.609999    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"]
Oct 30 18:35:51.610426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.610284    5746 cpu_manager.go:414] "containerMap: removing container" podUID="fc68792d-6dde-4398-a24d-eee8c143d22a" containerName="curl"
Oct 30 18:35:51.610426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.610300    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc68792d-6dde-4398-a24d-eee8c143d22a" containerName="curl"
Oct 30 18:35:51.610426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.610367    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc68792d-6dde-4398-a24d-eee8c143d22a" containerName="curl"
Oct 30 18:35:51.613330 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.613309    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"
Oct 30 18:35:51.629018 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.628996    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"]
Oct 30 18:35:51.639961 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.639943    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-network-policy-18e3625bcbbc44ff\"/\"default-dockercfg-w486b\""
Oct 30 18:35:51.644026 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.644009    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"
Oct 30 18:35:51.938700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:51.938628    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"]
Oct 30 18:35:51.941689 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:35:51.941656    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65339edd_0e05_46b7_a7d8_55d759eaef39.slice/crio-5cfbd14e1c6150701d0909a19fbbd1cf802d5a616dfece4940379a06ae6004cf WatchSource:0}: Error finding container 5cfbd14e1c6150701d0909a19fbbd1cf802d5a616dfece4940379a06ae6004cf: Status 404 returned error can't find the container with id 5cfbd14e1c6150701d0909a19fbbd1cf802d5a616dfece4940379a06ae6004cf
Oct 30 18:35:52.881200 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:52.881165    5746 generic.go:356] "Generic (PLEG): container finished" podID="65339edd-0e05-46b7-a7d8-55d759eaef39" containerID="e5671958b1509ab29d3648f2eeea27cf818a3b0a53cae4627c3ce22f9263d7a4" exitCode=0
Oct 30 18:35:52.881622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:52.881232    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c" event={"ID":"65339edd-0e05-46b7-a7d8-55d759eaef39","Type":"ContainerDied","Data":"e5671958b1509ab29d3648f2eeea27cf818a3b0a53cae4627c3ce22f9263d7a4"}
Oct 30 18:35:52.881622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:52.881265    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c" event={"ID":"65339edd-0e05-46b7-a7d8-55d759eaef39","Type":"ContainerStarted","Data":"5cfbd14e1c6150701d0909a19fbbd1cf802d5a616dfece4940379a06ae6004cf"}
Oct 30 18:35:54.026100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:54.026075    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"
Oct 30 18:35:54.891778 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:54.891751    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"
Oct 30 18:35:54.891986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:54.891788    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c" event={"ID":"65339edd-0e05-46b7-a7d8-55d759eaef39","Type":"ContainerDied","Data":"5cfbd14e1c6150701d0909a19fbbd1cf802d5a616dfece4940379a06ae6004cf"}
Oct 30 18:35:54.891986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:54.891826    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5cfbd14e1c6150701d0909a19fbbd1cf802d5a616dfece4940379a06ae6004cf"
Oct 30 18:35:55.736707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:55.736662    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"]
Oct 30 18:35:55.737228 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:55.737105    5746 cpu_manager.go:414] "containerMap: removing container" podUID="65339edd-0e05-46b7-a7d8-55d759eaef39" containerName="curl"
Oct 30 18:35:55.737228 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:55.737123    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65339edd-0e05-46b7-a7d8-55d759eaef39" containerName="curl"
Oct 30 18:35:55.737368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:55.737244    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="65339edd-0e05-46b7-a7d8-55d759eaef39" containerName="curl"
Oct 30 18:35:55.740103 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:55.740081    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"
Oct 30 18:35:55.756466 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:55.756442    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-network-policy-18e3625bcbbc44ff\"/\"default-dockercfg-w486b\""
Oct 30 18:35:55.757488 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:55.757465    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"]
Oct 30 18:35:55.760503 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:55.760481    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"
Oct 30 18:35:56.054114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:56.054044    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"]
Oct 30 18:35:56.057189 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:35:56.057164    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod892a0e88_c9f2_4b4d_8be5_046375e13828.slice/crio-e2c8e4993e58b7c472638a4c09b5ac447ec8cac858be6eef9321b30929e4d608 WatchSource:0}: Error finding container e2c8e4993e58b7c472638a4c09b5ac447ec8cac858be6eef9321b30929e4d608: Status 404 returned error can't find the container with id e2c8e4993e58b7c472638a4c09b5ac447ec8cac858be6eef9321b30929e4d608
Oct 30 18:35:56.902656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:56.902619    5746 generic.go:356] "Generic (PLEG): container finished" podID="892a0e88-c9f2-4b4d-8be5-046375e13828" containerID="6a9028373b2164f4d94d5712602ff64ed9a77cc58ba4cb443e5c2836f4dfa74e" exitCode=0
Oct 30 18:35:56.903062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:56.902665    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc" event={"ID":"892a0e88-c9f2-4b4d-8be5-046375e13828","Type":"ContainerDied","Data":"6a9028373b2164f4d94d5712602ff64ed9a77cc58ba4cb443e5c2836f4dfa74e"}
Oct 30 18:35:56.903062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:56.902698    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc" event={"ID":"892a0e88-c9f2-4b4d-8be5-046375e13828","Type":"ContainerStarted","Data":"e2c8e4993e58b7c472638a4c09b5ac447ec8cac858be6eef9321b30929e4d608"}
Oct 30 18:35:58.042401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:58.042377    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"
Oct 30 18:35:58.915457 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:58.915425    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc" event={"ID":"892a0e88-c9f2-4b4d-8be5-046375e13828","Type":"ContainerDied","Data":"e2c8e4993e58b7c472638a4c09b5ac447ec8cac858be6eef9321b30929e4d608"}
Oct 30 18:35:58.915457 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:58.915449    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"
Oct 30 18:35:58.915457 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:58.915463    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2c8e4993e58b7c472638a4c09b5ac447ec8cac858be6eef9321b30929e4d608"
Oct 30 18:35:59.868198 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:59.868164    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"]
Oct 30 18:35:59.868584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:59.868447    5746 cpu_manager.go:414] "containerMap: removing container" podUID="892a0e88-c9f2-4b4d-8be5-046375e13828" containerName="curl"
Oct 30 18:35:59.868584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:59.868457    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="892a0e88-c9f2-4b4d-8be5-046375e13828" containerName="curl"
Oct 30 18:35:59.868584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:59.868509    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="892a0e88-c9f2-4b4d-8be5-046375e13828" containerName="curl"
Oct 30 18:35:59.871287 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:59.871269    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"
Oct 30 18:35:59.888386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:59.888360    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"]
Oct 30 18:35:59.896431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:59.896412    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-network-policy-18e3625bcbbc44ff\"/\"default-dockercfg-w486b\""
Oct 30 18:35:59.902501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:35:59.902483    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"
Oct 30 18:36:00.198107 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:00.198034    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"]
Oct 30 18:36:00.201181 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:00.201125    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f8a1b8c_088e_4d87_a2f4_9b380dc33cbe.slice/crio-40398ecbd1c9deafce765569b1c3efc8e1d24929c96589b67eb50a6320483875 WatchSource:0}: Error finding container 40398ecbd1c9deafce765569b1c3efc8e1d24929c96589b67eb50a6320483875: Status 404 returned error can't find the container with id 40398ecbd1c9deafce765569b1c3efc8e1d24929c96589b67eb50a6320483875
Oct 30 18:36:00.925790 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:00.925755    5746 generic.go:356] "Generic (PLEG): container finished" podID="2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe" containerID="5df9097955aa33c267a93bc4c93d7fe0bf24c1a3d039828f6e72ab56a67c388f" exitCode=0
Oct 30 18:36:00.925790 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:00.925785    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708" event={"ID":"2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe","Type":"ContainerDied","Data":"5df9097955aa33c267a93bc4c93d7fe0bf24c1a3d039828f6e72ab56a67c388f"}
Oct 30 18:36:00.926389 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:00.925830    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708" event={"ID":"2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe","Type":"ContainerStarted","Data":"40398ecbd1c9deafce765569b1c3efc8e1d24929c96589b67eb50a6320483875"}
Oct 30 18:36:02.052382 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:02.052354    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"
Oct 30 18:36:02.936740 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:02.936710    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"
Oct 30 18:36:02.937006 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:02.936710    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708" event={"ID":"2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe","Type":"ContainerDied","Data":"40398ecbd1c9deafce765569b1c3efc8e1d24929c96589b67eb50a6320483875"}
Oct 30 18:36:02.937006 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:02.936828    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40398ecbd1c9deafce765569b1c3efc8e1d24929c96589b67eb50a6320483875"
Oct 30 18:36:03.984690 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:03.984657    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"]
Oct 30 18:36:03.985101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:03.984942    5746 cpu_manager.go:414] "containerMap: removing container" podUID="2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe" containerName="curl"
Oct 30 18:36:03.985101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:03.984953    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe" containerName="curl"
Oct 30 18:36:03.985101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:03.985013    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe" containerName="curl"
Oct 30 18:36:03.987717 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:03.987699    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"
Oct 30 18:36:04.009072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:04.009043    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"]
Oct 30 18:36:04.016852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:04.016834    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-network-policy-18e3625bcbbc44ff\"/\"default-dockercfg-w486b\""
Oct 30 18:36:04.018301 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:04.018284    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"
Oct 30 18:36:04.313404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:04.313285    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"]
Oct 30 18:36:04.316254 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:04.316219    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod806ea03c_ef8b_4cac_bc65_234ac96e8bc7.slice/crio-3ba30a54fbf54a0078f933dc8c67f50b2b0e569bd48d0409ef66ab09672dc165 WatchSource:0}: Error finding container 3ba30a54fbf54a0078f933dc8c67f50b2b0e569bd48d0409ef66ab09672dc165: Status 404 returned error can't find the container with id 3ba30a54fbf54a0078f933dc8c67f50b2b0e569bd48d0409ef66ab09672dc165
Oct 30 18:36:04.947317 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:04.947274    5746 generic.go:356] "Generic (PLEG): container finished" podID="806ea03c-ef8b-4cac-bc65-234ac96e8bc7" containerID="f46204425f7e2a4823216afcad0a16ca3740ed28a4d86e928c9f82afc57caa5a" exitCode=0
Oct 30 18:36:04.947504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:04.947340    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc" event={"ID":"806ea03c-ef8b-4cac-bc65-234ac96e8bc7","Type":"ContainerDied","Data":"f46204425f7e2a4823216afcad0a16ca3740ed28a4d86e928c9f82afc57caa5a"}
Oct 30 18:36:04.947504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:04.947373    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc" event={"ID":"806ea03c-ef8b-4cac-bc65-234ac96e8bc7","Type":"ContainerStarted","Data":"3ba30a54fbf54a0078f933dc8c67f50b2b0e569bd48d0409ef66ab09672dc165"}
Oct 30 18:36:06.079266 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:06.079243    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"
Oct 30 18:36:06.957779 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:06.957747    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"
Oct 30 18:36:06.958022 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:06.957751    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc" event={"ID":"806ea03c-ef8b-4cac-bc65-234ac96e8bc7","Type":"ContainerDied","Data":"3ba30a54fbf54a0078f933dc8c67f50b2b0e569bd48d0409ef66ab09672dc165"}
Oct 30 18:36:06.958022 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:06.957862    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ba30a54fbf54a0078f933dc8c67f50b2b0e569bd48d0409ef66ab09672dc165"
Oct 30 18:36:08.100619 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.100573    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"]
Oct 30 18:36:08.101164 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.101011    5746 cpu_manager.go:414] "containerMap: removing container" podUID="806ea03c-ef8b-4cac-bc65-234ac96e8bc7" containerName="curl"
Oct 30 18:36:08.101164 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.101029    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="806ea03c-ef8b-4cac-bc65-234ac96e8bc7" containerName="curl"
Oct 30 18:36:08.101164 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.101120    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="806ea03c-ef8b-4cac-bc65-234ac96e8bc7" containerName="curl"
Oct 30 18:36:08.104058 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.104038    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"
Oct 30 18:36:08.121110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.121081    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"]
Oct 30 18:36:08.131949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.131930    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-network-policy-18e3625bcbbc44ff\"/\"default-dockercfg-w486b\""
Oct 30 18:36:08.135420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.135404    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"
Oct 30 18:36:08.430130 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.430056    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"]
Oct 30 18:36:08.433380 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:08.433351    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode751edb8_b5b3_4e2f_911c_83b113b61899.slice/crio-7fbf49243e218e4b56177e617535565a1d8c79e1630786ad4f48662c9ffbeee8 WatchSource:0}: Error finding container 7fbf49243e218e4b56177e617535565a1d8c79e1630786ad4f48662c9ffbeee8: Status 404 returned error can't find the container with id 7fbf49243e218e4b56177e617535565a1d8c79e1630786ad4f48662c9ffbeee8
Oct 30 18:36:08.968673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.968636    5746 generic.go:356] "Generic (PLEG): container finished" podID="e751edb8-b5b3-4e2f-911c-83b113b61899" containerID="cd6e0bbb15ef668fb249844dada08a6293eaaa666ccb86d59bc5125ee41ed218" exitCode=0
Oct 30 18:36:08.968879 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.968693    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9" event={"ID":"e751edb8-b5b3-4e2f-911c-83b113b61899","Type":"ContainerDied","Data":"cd6e0bbb15ef668fb249844dada08a6293eaaa666ccb86d59bc5125ee41ed218"}
Oct 30 18:36:08.968879 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:08.968744    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9" event={"ID":"e751edb8-b5b3-4e2f-911c-83b113b61899","Type":"ContainerStarted","Data":"7fbf49243e218e4b56177e617535565a1d8c79e1630786ad4f48662c9ffbeee8"}
Oct 30 18:36:10.103255 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:10.103232    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"
Oct 30 18:36:10.979514 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:10.979485    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"
Oct 30 18:36:10.979701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:10.979516    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9" event={"ID":"e751edb8-b5b3-4e2f-911c-83b113b61899","Type":"ContainerDied","Data":"7fbf49243e218e4b56177e617535565a1d8c79e1630786ad4f48662c9ffbeee8"}
Oct 30 18:36:10.979701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:10.979557    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fbf49243e218e4b56177e617535565a1d8c79e1630786ad4f48662c9ffbeee8"
Oct 30 18:36:12.216686 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.216650    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"]
Oct 30 18:36:12.217095 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.216958    5746 cpu_manager.go:414] "containerMap: removing container" podUID="e751edb8-b5b3-4e2f-911c-83b113b61899" containerName="curl"
Oct 30 18:36:12.217095 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.216970    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e751edb8-b5b3-4e2f-911c-83b113b61899" containerName="curl"
Oct 30 18:36:12.217095 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.217022    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="e751edb8-b5b3-4e2f-911c-83b113b61899" containerName="curl"
Oct 30 18:36:12.219945 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.219930    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"
Oct 30 18:36:12.237300 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.237276    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"]
Oct 30 18:36:12.248122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.248097    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-network-policy-18e3625bcbbc44ff\"/\"default-dockercfg-w486b\""
Oct 30 18:36:12.250181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.250162    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"
Oct 30 18:36:12.544392 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.544224    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"]
Oct 30 18:36:12.547083 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:12.547054    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf92286dc_91a4_4622_b15f_ffbf59d5008f.slice/crio-591bc1fa9be3012f53384a63de5498027673af2393b7eb082fc348a2730f4569 WatchSource:0}: Error finding container 591bc1fa9be3012f53384a63de5498027673af2393b7eb082fc348a2730f4569: Status 404 returned error can't find the container with id 591bc1fa9be3012f53384a63de5498027673af2393b7eb082fc348a2730f4569
Oct 30 18:36:12.789826 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:36:12.789789    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:36:12.989544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.989502    5746 generic.go:356] "Generic (PLEG): container finished" podID="f92286dc-91a4-4622-b15f-ffbf59d5008f" containerID="645e4c67eea22676e68897b1997eebcb45a997718ae1a7c327c4a729d11ad5e4" exitCode=0
Oct 30 18:36:12.989718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.989563    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb" event={"ID":"f92286dc-91a4-4622-b15f-ffbf59d5008f","Type":"ContainerDied","Data":"645e4c67eea22676e68897b1997eebcb45a997718ae1a7c327c4a729d11ad5e4"}
Oct 30 18:36:12.989718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:12.989635    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb" event={"ID":"f92286dc-91a4-4622-b15f-ffbf59d5008f","Type":"ContainerStarted","Data":"591bc1fa9be3012f53384a63de5498027673af2393b7eb082fc348a2730f4569"}
Oct 30 18:36:14.124367 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.124342    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"
Oct 30 18:36:14.316933 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.316830    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"]
Oct 30 18:36:14.317149 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.317136    5746 cpu_manager.go:414] "containerMap: removing container" podUID="f92286dc-91a4-4622-b15f-ffbf59d5008f" containerName="curl"
Oct 30 18:36:14.317188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.317150    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f92286dc-91a4-4622-b15f-ffbf59d5008f" containerName="curl"
Oct 30 18:36:14.317234 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.317206    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="f92286dc-91a4-4622-b15f-ffbf59d5008f" containerName="curl"
Oct 30 18:36:14.319973 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.319956    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"
Oct 30 18:36:14.330566 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.330548    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"
Oct 30 18:36:14.336702 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.336634    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"]
Oct 30 18:36:14.627478 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.627433    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"]
Oct 30 18:36:14.630490 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:14.630455    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c801ba6_94b8_44ed_91d9_bf297e42396b.slice/crio-002d5050e693e337df2b9a56f9c94c3a100fff2c2f7a0e1c417605b676bb745a WatchSource:0}: Error finding container 002d5050e693e337df2b9a56f9c94c3a100fff2c2f7a0e1c417605b676bb745a: Status 404 returned error can't find the container with id 002d5050e693e337df2b9a56f9c94c3a100fff2c2f7a0e1c417605b676bb745a
Oct 30 18:36:14.999580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.999479    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb" event={"ID":"f92286dc-91a4-4622-b15f-ffbf59d5008f","Type":"ContainerDied","Data":"591bc1fa9be3012f53384a63de5498027673af2393b7eb082fc348a2730f4569"}
Oct 30 18:36:14.999580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.999519    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="591bc1fa9be3012f53384a63de5498027673af2393b7eb082fc348a2730f4569"
Oct 30 18:36:14.999580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:14.999494    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"
Oct 30 18:36:15.001004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:15.000980    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20" event={"ID":"1c801ba6-94b8-44ed-91d9-bf297e42396b","Type":"ContainerStarted","Data":"1b3f9fc33a8be132a30517429931801ea42ed0ccb9d5f4e1b0ec7bf01c387769"}
Oct 30 18:36:15.001004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:15.001014    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20" event={"ID":"1c801ba6-94b8-44ed-91d9-bf297e42396b","Type":"ContainerStarted","Data":"002d5050e693e337df2b9a56f9c94c3a100fff2c2f7a0e1c417605b676bb745a"}
Oct 30 18:36:15.039850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:15.039804    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20" podStartSLOduration=1.039791718 podStartE2EDuration="1.039791718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:15.038939837 +0000 UTC m=+2345.205757964" watchObservedRunningTime="2026-10-30 18:36:15.039791718 +0000 UTC m=+2345.206609845"
Oct 30 18:36:23.037145 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:23.037110    5746 generic.go:356] "Generic (PLEG): container finished" podID="1c801ba6-94b8-44ed-91d9-bf297e42396b" containerID="1b3f9fc33a8be132a30517429931801ea42ed0ccb9d5f4e1b0ec7bf01c387769" exitCode=0
Oct 30 18:36:23.037656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:23.037188    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20" event={"ID":"1c801ba6-94b8-44ed-91d9-bf297e42396b","Type":"ContainerDied","Data":"1b3f9fc33a8be132a30517429931801ea42ed0ccb9d5f4e1b0ec7bf01c387769"}
Oct 30 18:36:24.168679 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.168649    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"
Oct 30 18:36:24.482832 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.482798    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"]
Oct 30 18:36:24.483133 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.483121    5746 cpu_manager.go:414] "containerMap: removing container" podUID="1c801ba6-94b8-44ed-91d9-bf297e42396b" containerName="curl"
Oct 30 18:36:24.483179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.483135    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c801ba6-94b8-44ed-91d9-bf297e42396b" containerName="curl"
Oct 30 18:36:24.483213 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.483199    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c801ba6-94b8-44ed-91d9-bf297e42396b" containerName="curl"
Oct 30 18:36:24.486173 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.486153    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"
Oct 30 18:36:24.496400 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.496379    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"
Oct 30 18:36:24.501967 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.501941    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"]
Oct 30 18:36:24.792830 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:24.792715    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"]
Oct 30 18:36:24.795710 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:24.795684    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cc763e0_cae5_45ff_a1bf_f696b5c1dd62.slice/crio-137eb6ec62b777aee6480deacab3633a1735eaf95831a533a565dfc96f9e48a3 WatchSource:0}: Error finding container 137eb6ec62b777aee6480deacab3633a1735eaf95831a533a565dfc96f9e48a3: Status 404 returned error can't find the container with id 137eb6ec62b777aee6480deacab3633a1735eaf95831a533a565dfc96f9e48a3
Oct 30 18:36:25.047398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:25.047313    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"
Oct 30 18:36:25.047398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:25.047325    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20" event={"ID":"1c801ba6-94b8-44ed-91d9-bf297e42396b","Type":"ContainerDied","Data":"002d5050e693e337df2b9a56f9c94c3a100fff2c2f7a0e1c417605b676bb745a"}
Oct 30 18:36:25.047398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:25.047363    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="002d5050e693e337df2b9a56f9c94c3a100fff2c2f7a0e1c417605b676bb745a"
Oct 30 18:36:25.048973 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:25.048947    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4" event={"ID":"8cc763e0-cae5-45ff-a1bf-f696b5c1dd62","Type":"ContainerStarted","Data":"221737772d1e3e03594a0405d3705b6500b2c5f2ea57a83bf570eac89cfb00de"}
Oct 30 18:36:25.049095 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:25.048982    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4" event={"ID":"8cc763e0-cae5-45ff-a1bf-f696b5c1dd62","Type":"ContainerStarted","Data":"137eb6ec62b777aee6480deacab3633a1735eaf95831a533a565dfc96f9e48a3"}
Oct 30 18:36:25.088430 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:25.088379    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4" podStartSLOduration=1.088366585 podStartE2EDuration="1.088366585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:25.087578929 +0000 UTC m=+2355.254397057" watchObservedRunningTime="2026-10-30 18:36:25.088366585 +0000 UTC m=+2355.255184728"
Oct 30 18:36:33.085956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:33.085900    5746 generic.go:356] "Generic (PLEG): container finished" podID="8cc763e0-cae5-45ff-a1bf-f696b5c1dd62" containerID="221737772d1e3e03594a0405d3705b6500b2c5f2ea57a83bf570eac89cfb00de" exitCode=0
Oct 30 18:36:33.086401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:33.085973    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4" event={"ID":"8cc763e0-cae5-45ff-a1bf-f696b5c1dd62","Type":"ContainerDied","Data":"221737772d1e3e03594a0405d3705b6500b2c5f2ea57a83bf570eac89cfb00de"}
Oct 30 18:36:34.217102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.217076    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"
Oct 30 18:36:34.643251 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.643215    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"]
Oct 30 18:36:34.643532 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.643516    5746 cpu_manager.go:414] "containerMap: removing container" podUID="8cc763e0-cae5-45ff-a1bf-f696b5c1dd62" containerName="curl"
Oct 30 18:36:34.643611 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.643533    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cc763e0-cae5-45ff-a1bf-f696b5c1dd62" containerName="curl"
Oct 30 18:36:34.643611 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.643601    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cc763e0-cae5-45ff-a1bf-f696b5c1dd62" containerName="curl"
Oct 30 18:36:34.646510 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.646494    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"
Oct 30 18:36:34.656941 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.656925    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"
Oct 30 18:36:34.665170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.665142    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"]
Oct 30 18:36:34.954869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:34.954847    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"]
Oct 30 18:36:34.957564 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:34.957535    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39b64939_e788_4a14_9987_c246f6f89167.slice/crio-2b680b3c779d95fb6a9a30803a58efe62b0b0371bad80c161f4aa8421bfaa663 WatchSource:0}: Error finding container 2b680b3c779d95fb6a9a30803a58efe62b0b0371bad80c161f4aa8421bfaa663: Status 404 returned error can't find the container with id 2b680b3c779d95fb6a9a30803a58efe62b0b0371bad80c161f4aa8421bfaa663
Oct 30 18:36:35.096879 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:35.096845    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b" event={"ID":"39b64939-e788-4a14-9987-c246f6f89167","Type":"ContainerStarted","Data":"e3f06717abab75a6591162b0df3615e2d7b433c73d4d71c801cc03d535ff1afb"}
Oct 30 18:36:35.097080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:35.096890    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b" event={"ID":"39b64939-e788-4a14-9987-c246f6f89167","Type":"ContainerStarted","Data":"2b680b3c779d95fb6a9a30803a58efe62b0b0371bad80c161f4aa8421bfaa663"}
Oct 30 18:36:35.098129 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:35.098111    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"
Oct 30 18:36:35.098211 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:35.098154    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4" event={"ID":"8cc763e0-cae5-45ff-a1bf-f696b5c1dd62","Type":"ContainerDied","Data":"137eb6ec62b777aee6480deacab3633a1735eaf95831a533a565dfc96f9e48a3"}
Oct 30 18:36:35.098211 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:35.098182    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="137eb6ec62b777aee6480deacab3633a1735eaf95831a533a565dfc96f9e48a3"
Oct 30 18:36:35.135694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:35.135647    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b" podStartSLOduration=1.135635791 podStartE2EDuration="1.135635791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:35.13505659 +0000 UTC m=+2365.301874721" watchObservedRunningTime="2026-10-30 18:36:35.135635791 +0000 UTC m=+2365.302453915"
Oct 30 18:36:42.789368 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:36:42.789281    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:36:43.134415 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:43.134384    5746 generic.go:356] "Generic (PLEG): container finished" podID="39b64939-e788-4a14-9987-c246f6f89167" containerID="e3f06717abab75a6591162b0df3615e2d7b433c73d4d71c801cc03d535ff1afb" exitCode=0
Oct 30 18:36:43.134591 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:43.134444    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b" event={"ID":"39b64939-e788-4a14-9987-c246f6f89167","Type":"ContainerDied","Data":"e3f06717abab75a6591162b0df3615e2d7b433c73d4d71c801cc03d535ff1afb"}
Oct 30 18:36:44.274074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:44.274046    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"
Oct 30 18:36:44.818426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:44.818395    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"]
Oct 30 18:36:44.818693 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:44.818682    5746 cpu_manager.go:414] "containerMap: removing container" podUID="39b64939-e788-4a14-9987-c246f6f89167" containerName="curl"
Oct 30 18:36:44.818742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:44.818694    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39b64939-e788-4a14-9987-c246f6f89167" containerName="curl"
Oct 30 18:36:44.818801 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:44.818748    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="39b64939-e788-4a14-9987-c246f6f89167" containerName="curl"
Oct 30 18:36:44.821696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:44.821678    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"
Oct 30 18:36:44.832409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:44.832388    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"
Oct 30 18:36:44.838953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:44.838926    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"]
Oct 30 18:36:45.127089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:45.127065    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"]
Oct 30 18:36:45.129703 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:45.129675    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd87b0f0b_c3e5_438a_9e8a_c591fa7e9c10.slice/crio-043a6d967d9fd2ba0f7ee64391cb58ede0a6d00df2bb5aca1ec457955d9e1ba4 WatchSource:0}: Error finding container 043a6d967d9fd2ba0f7ee64391cb58ede0a6d00df2bb5aca1ec457955d9e1ba4: Status 404 returned error can't find the container with id 043a6d967d9fd2ba0f7ee64391cb58ede0a6d00df2bb5aca1ec457955d9e1ba4
Oct 30 18:36:45.145870 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:45.145843    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb" event={"ID":"d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10","Type":"ContainerStarted","Data":"043a6d967d9fd2ba0f7ee64391cb58ede0a6d00df2bb5aca1ec457955d9e1ba4"}
Oct 30 18:36:45.147279 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:45.147262    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"
Oct 30 18:36:45.147279 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:45.147269    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b" event={"ID":"39b64939-e788-4a14-9987-c246f6f89167","Type":"ContainerDied","Data":"2b680b3c779d95fb6a9a30803a58efe62b0b0371bad80c161f4aa8421bfaa663"}
Oct 30 18:36:45.147413 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:45.147295    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b680b3c779d95fb6a9a30803a58efe62b0b0371bad80c161f4aa8421bfaa663"
Oct 30 18:36:46.154430 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:46.154395    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb" event={"ID":"d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10","Type":"ContainerStarted","Data":"bee778dad7f8ad4517d580f706c35405e988c44f80e5a540d6a68736e55e9111"}
Oct 30 18:36:46.193039 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:46.192990    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb" podStartSLOduration=2.192977013 podStartE2EDuration="2.192977013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:46.192077798 +0000 UTC m=+2376.358895936" watchObservedRunningTime="2026-10-30 18:36:46.192977013 +0000 UTC m=+2376.359795176"
Oct 30 18:36:53.187196 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:53.187160    5746 generic.go:356] "Generic (PLEG): container finished" podID="d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10" containerID="bee778dad7f8ad4517d580f706c35405e988c44f80e5a540d6a68736e55e9111" exitCode=0
Oct 30 18:36:53.187638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:53.187230    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb" event={"ID":"d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10","Type":"ContainerDied","Data":"bee778dad7f8ad4517d580f706c35405e988c44f80e5a540d6a68736e55e9111"}
Oct 30 18:36:54.316204 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:54.316180    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"
Oct 30 18:36:54.993627 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:54.993590    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"]
Oct 30 18:36:54.993901 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:54.993889    5746 cpu_manager.go:414] "containerMap: removing container" podUID="d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10" containerName="curl"
Oct 30 18:36:54.993970 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:54.993920    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10" containerName="curl"
Oct 30 18:36:54.994044 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:54.993991    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10" containerName="curl"
Oct 30 18:36:54.996989 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:54.996969    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"
Oct 30 18:36:55.007094 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:55.007077    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"
Oct 30 18:36:55.014565 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:55.014544    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"]
Oct 30 18:36:55.200136 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:55.200103    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"
Oct 30 18:36:55.200303 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:55.200106    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb" event={"ID":"d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10","Type":"ContainerDied","Data":"043a6d967d9fd2ba0f7ee64391cb58ede0a6d00df2bb5aca1ec457955d9e1ba4"}
Oct 30 18:36:55.200303 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:55.200217    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="043a6d967d9fd2ba0f7ee64391cb58ede0a6d00df2bb5aca1ec457955d9e1ba4"
Oct 30 18:36:55.304323 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:55.304247    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"]
Oct 30 18:36:55.307346 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:55.307317    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5cb4d20_3159_454b_9a1c_3714fba15242.slice/crio-998165f1fe80a1a3f0502f514de7e7a1d912efeb61a074058ef90474895c3bf4 WatchSource:0}: Error finding container 998165f1fe80a1a3f0502f514de7e7a1d912efeb61a074058ef90474895c3bf4: Status 404 returned error can't find the container with id 998165f1fe80a1a3f0502f514de7e7a1d912efeb61a074058ef90474895c3bf4
Oct 30 18:36:56.208043 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:56.208008    5746 generic.go:356] "Generic (PLEG): container finished" podID="a5cb4d20-3159-454b-9a1c-3714fba15242" containerID="fd22393f50a4fb89a49e8182453e6256f16c73f44a86b93d3aa61b479259c8f7" exitCode=0
Oct 30 18:36:56.208437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:56.208096    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c" event={"ID":"a5cb4d20-3159-454b-9a1c-3714fba15242","Type":"ContainerDied","Data":"fd22393f50a4fb89a49e8182453e6256f16c73f44a86b93d3aa61b479259c8f7"}
Oct 30 18:36:56.208437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:56.208140    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c" event={"ID":"a5cb4d20-3159-454b-9a1c-3714fba15242","Type":"ContainerStarted","Data":"998165f1fe80a1a3f0502f514de7e7a1d912efeb61a074058ef90474895c3bf4"}
Oct 30 18:36:57.336290 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:57.336266    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"
Oct 30 18:36:58.218429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:58.218404    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"
Oct 30 18:36:58.218599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:58.218456    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c" event={"ID":"a5cb4d20-3159-454b-9a1c-3714fba15242","Type":"ContainerDied","Data":"998165f1fe80a1a3f0502f514de7e7a1d912efeb61a074058ef90474895c3bf4"}
Oct 30 18:36:58.218599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:58.218498    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="998165f1fe80a1a3f0502f514de7e7a1d912efeb61a074058ef90474895c3bf4"
Oct 30 18:36:59.108417 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:59.108381    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"]
Oct 30 18:36:59.108900 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:59.108801    5746 cpu_manager.go:414] "containerMap: removing container" podUID="a5cb4d20-3159-454b-9a1c-3714fba15242" containerName="curl"
Oct 30 18:36:59.108900 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:59.108818    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5cb4d20-3159-454b-9a1c-3714fba15242" containerName="curl"
Oct 30 18:36:59.109067 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:59.108933    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5cb4d20-3159-454b-9a1c-3714fba15242" containerName="curl"
Oct 30 18:36:59.111851 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:59.111832    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"
Oct 30 18:36:59.122240 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:59.122221    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"
Oct 30 18:36:59.129475 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:59.129452    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"]
Oct 30 18:36:59.417319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:36:59.417248    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"]
Oct 30 18:36:59.420147 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:36:59.420115    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ee527e8_b56a_49b6_b638_556fe61760e2.slice/crio-5a54326d2765ec7ef2132692204e87380c081fdd2f5a4b35cb993762e744ad35 WatchSource:0}: Error finding container 5a54326d2765ec7ef2132692204e87380c081fdd2f5a4b35cb993762e744ad35: Status 404 returned error can't find the container with id 5a54326d2765ec7ef2132692204e87380c081fdd2f5a4b35cb993762e744ad35
Oct 30 18:37:00.228979 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:00.228942    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231" event={"ID":"9ee527e8-b56a-49b6-b638-556fe61760e2","Type":"ContainerStarted","Data":"808288db1cb7fc9bc6cb6f66c8e72d2b26b7e59886b142c5826c5bdcb34a552e"}
Oct 30 18:37:00.228979 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:00.228983    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231" event={"ID":"9ee527e8-b56a-49b6-b638-556fe61760e2","Type":"ContainerStarted","Data":"5a54326d2765ec7ef2132692204e87380c081fdd2f5a4b35cb993762e744ad35"}
Oct 30 18:37:00.268626 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:00.268579    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231" podStartSLOduration=1.268566428 podStartE2EDuration="1.268566428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:00.268052369 +0000 UTC m=+2390.434870498" watchObservedRunningTime="2026-10-30 18:37:00.268566428 +0000 UTC m=+2390.435384555"
Oct 30 18:37:05.253379 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:05.253343    5746 generic.go:356] "Generic (PLEG): container finished" podID="9ee527e8-b56a-49b6-b638-556fe61760e2" containerID="808288db1cb7fc9bc6cb6f66c8e72d2b26b7e59886b142c5826c5bdcb34a552e" exitCode=0
Oct 30 18:37:05.253811 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:05.253414    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231" event={"ID":"9ee527e8-b56a-49b6-b638-556fe61760e2","Type":"ContainerDied","Data":"808288db1cb7fc9bc6cb6f66c8e72d2b26b7e59886b142c5826c5bdcb34a552e"}
Oct 30 18:37:06.392477 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:06.392449    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"
Oct 30 18:37:07.269332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.269299    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"
Oct 30 18:37:07.269523 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.269298    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231" event={"ID":"9ee527e8-b56a-49b6-b638-556fe61760e2","Type":"ContainerDied","Data":"5a54326d2765ec7ef2132692204e87380c081fdd2f5a4b35cb993762e744ad35"}
Oct 30 18:37:07.269523 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.269428    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a54326d2765ec7ef2132692204e87380c081fdd2f5a4b35cb993762e744ad35"
Oct 30 18:37:07.271793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.271771    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"]
Oct 30 18:37:07.272088 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.272075    5746 cpu_manager.go:414] "containerMap: removing container" podUID="9ee527e8-b56a-49b6-b638-556fe61760e2" containerName="curl"
Oct 30 18:37:07.272135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.272089    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ee527e8-b56a-49b6-b638-556fe61760e2" containerName="curl"
Oct 30 18:37:07.272175 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.272146    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ee527e8-b56a-49b6-b638-556fe61760e2" containerName="curl"
Oct 30 18:37:07.275223 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.275179    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"
Oct 30 18:37:07.293758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.293703    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"]
Oct 30 18:37:07.303985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.303966    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:37:07.449090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.449053    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-certificate\" (UniqueName: \"kubernetes.io/secret/6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc-client-certificate\") pod \"targetallocator-mtls-allowed-18e3628b5540c61f\" (UID: \"6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc\") " pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"
Oct 30 18:37:07.549549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.549457    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-certificate\" (UniqueName: \"kubernetes.io/secret/6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc-client-certificate\") pod \"targetallocator-mtls-allowed-18e3628b5540c61f\" (UID: \"6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc\") " pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"
Oct 30 18:37:07.552246 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.552224    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-certificate\" (UniqueName: \"kubernetes.io/secret/6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc-client-certificate\") pod \"targetallocator-mtls-allowed-18e3628b5540c61f\" (UID: \"6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc\") " pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"
Oct 30 18:37:07.587004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.586965    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"
Oct 30 18:37:07.884223 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:07.884194    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"]
Oct 30 18:37:07.887563 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:37:07.887540    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c825a3c_97c7_48f7_85b7_7bce9f2aa9cc.slice/crio-689c7cac556f15b66fcf4dd13159a1f44cd48b0b19f2b1f6958b737d99d10ae1 WatchSource:0}: Error finding container 689c7cac556f15b66fcf4dd13159a1f44cd48b0b19f2b1f6958b737d99d10ae1: Status 404 returned error can't find the container with id 689c7cac556f15b66fcf4dd13159a1f44cd48b0b19f2b1f6958b737d99d10ae1
Oct 30 18:37:08.275134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:08.275043    5746 generic.go:356] "Generic (PLEG): container finished" podID="6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc" containerID="24875b1351a2ee7475a00fa816225dd36de91392eb1b3cf8fc648d13b079f4b6" exitCode=0
Oct 30 18:37:08.275134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:08.275125    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f" event={"ID":"6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc","Type":"ContainerDied","Data":"24875b1351a2ee7475a00fa816225dd36de91392eb1b3cf8fc648d13b079f4b6"}
Oct 30 18:37:08.275326 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:08.275164    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f" event={"ID":"6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc","Type":"ContainerStarted","Data":"689c7cac556f15b66fcf4dd13159a1f44cd48b0b19f2b1f6958b737d99d10ae1"}
Oct 30 18:37:09.409666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:09.409642    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"
Oct 30 18:37:09.467154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:09.467122    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc-client-certificate\" (UniqueName: \"kubernetes.io/secret/6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc-client-certificate\") pod \"6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc\" (UID: \"6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc\") "
Oct 30 18:37:09.469238 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:09.469215    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc-client-certificate" pod "6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc" (UID: "6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc"). InnerVolumeSpecName "client-certificate". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:37:09.568512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:09.568434    5746 reconciler_common.go:299] "Volume detached for volume \"client-certificate\" (UniqueName: \"kubernetes.io/secret/6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc-client-certificate\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:10.286135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:10.286047    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"
Oct 30 18:37:10.286135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:10.286053    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f" event={"ID":"6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc","Type":"ContainerDied","Data":"689c7cac556f15b66fcf4dd13159a1f44cd48b0b19f2b1f6958b737d99d10ae1"}
Oct 30 18:37:10.286135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:10.286096    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="689c7cac556f15b66fcf4dd13159a1f44cd48b0b19f2b1f6958b737d99d10ae1"
Oct 30 18:37:11.388118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:11.388085    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"]
Oct 30 18:37:11.388500 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:11.388392    5746 cpu_manager.go:414] "containerMap: removing container" podUID="6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc" containerName="curl"
Oct 30 18:37:11.388500 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:11.388402    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc" containerName="curl"
Oct 30 18:37:11.388500 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:11.388454    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc" containerName="curl"
Oct 30 18:37:11.391391 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:11.391374    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"
Oct 30 18:37:11.401638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:11.401618    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"
Oct 30 18:37:11.408953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:11.408934    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"]
Oct 30 18:37:11.697022 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:11.696793    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"]
Oct 30 18:37:11.699517 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:37:11.699490    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24b1fa37_26d8_4d94_aca7_130db2eefa73.slice/crio-0e88bef37715e035b28c5479558a164742b80c1bb523ae42642e610795ef19d0 WatchSource:0}: Error finding container 0e88bef37715e035b28c5479558a164742b80c1bb523ae42642e610795ef19d0: Status 404 returned error can't find the container with id 0e88bef37715e035b28c5479558a164742b80c1bb523ae42642e610795ef19d0
Oct 30 18:37:12.296188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:12.296151    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69" event={"ID":"24b1fa37-26d8-4d94-aca7-130db2eefa73","Type":"ContainerStarted","Data":"f37eca64566601a36df7016070d61d616da35314be8e4cb14d4c5a7f4a8bc738"}
Oct 30 18:37:12.296188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:12.296193    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69" event={"ID":"24b1fa37-26d8-4d94-aca7-130db2eefa73","Type":"ContainerStarted","Data":"0e88bef37715e035b28c5479558a164742b80c1bb523ae42642e610795ef19d0"}
Oct 30 18:37:12.334466 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:12.334421    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69" podStartSLOduration=1.334408892 podStartE2EDuration="1.334408892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:12.333831072 +0000 UTC m=+2402.500649203" watchObservedRunningTime="2026-10-30 18:37:12.334408892 +0000 UTC m=+2402.501227049"
Oct 30 18:37:12.789274 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:12.789242    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:37:17.323872 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:17.323839    5746 generic.go:356] "Generic (PLEG): container finished" podID="24b1fa37-26d8-4d94-aca7-130db2eefa73" containerID="f37eca64566601a36df7016070d61d616da35314be8e4cb14d4c5a7f4a8bc738" exitCode=0
Oct 30 18:37:17.324286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:17.323922    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69" event={"ID":"24b1fa37-26d8-4d94-aca7-130db2eefa73","Type":"ContainerDied","Data":"f37eca64566601a36df7016070d61d616da35314be8e4cb14d4c5a7f4a8bc738"}
Oct 30 18:37:18.456342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:18.456319    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"
Oct 30 18:37:19.334451 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.334419    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"
Oct 30 18:37:19.334640 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.334460    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69" event={"ID":"24b1fa37-26d8-4d94-aca7-130db2eefa73","Type":"ContainerDied","Data":"0e88bef37715e035b28c5479558a164742b80c1bb523ae42642e610795ef19d0"}
Oct 30 18:37:19.334640 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.334494    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e88bef37715e035b28c5479558a164742b80c1bb523ae42642e610795ef19d0"
Oct 30 18:37:19.476124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.476087    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"]
Oct 30 18:37:19.482265 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.482235    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-untrusted-denied-18e3628c4aa33a69"]
Oct 30 18:37:19.500489 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.500462    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"]
Oct 30 18:37:19.504814 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.504794    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/targetallocator-mtls-allowed-18e3628b5540c61f"]
Oct 30 18:37:19.533350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.533323    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"]
Oct 30 18:37:19.534265 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.534246    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/targetallocator-untrusted-denied-18e362896ec6d231"]
Oct 30 18:37:19.547990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.547964    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"]
Oct 30 18:37:19.551770 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.551751    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/targetallocator-collector-allowed-18e3628878b82a1c"]
Oct 30 18:37:19.570812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.570787    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"]
Oct 30 18:37:19.572742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.572721    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-wrong-role-denied-18e362861a532ccb"]
Oct 30 18:37:19.590956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.590885    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"]
Oct 30 18:37:19.597377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.597355    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-role-only-denied-18e36283bc8c6d3b"]
Oct 30 18:37:19.616252 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.616231    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"]
Oct 30 18:37:19.620660 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.620641    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-part-only-denied-18e362815ecb0fa4"]
Oct 30 18:37:19.639108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.639087    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"]
Oct 30 18:37:19.643601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.643581    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-unlabeled-denied-18e3627f01151a20"]
Oct 30 18:37:19.661815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.661791    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"]
Oct 30 18:37:19.666040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.666023    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-inferenceservice-allowed-18e3627e83e2e2eb"]
Oct 30 18:37:19.687108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.687084    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"]
Oct 30 18:37:19.691637 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.691613    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-scheduler-allowed-18e3627d8e702ce9"]
Oct 30 18:37:19.710139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.710119    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"]
Oct 30 18:37:19.714262 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.714242    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-prefill-allowed-18e3627c98f263cc"]
Oct 30 18:37:19.732732 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.732712    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"]
Oct 30 18:37:19.736842 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.736822    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-prefill-allowed-18e3627ba2cd1708"]
Oct 30 18:37:19.755268 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.755245    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"]
Oct 30 18:37:19.759543 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.759524    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-leader-allowed-18e3627aacdd20cc"]
Oct 30 18:37:19.778385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.778365    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"]
Oct 30 18:37:19.786877 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.786858    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-worker-allowed-18e36279b6b4482c"]
Oct 30 18:37:19.805721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.805700    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"]
Oct 30 18:37:19.810162 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.810145    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-prefill-allowed-18e36278c159cb02"]
Oct 30 18:37:19.828877 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.828850    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"]
Oct 30 18:37:19.832703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.832684    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-network-policy-18e3625bcbbc44ff/collector-otlp-workload-allowed-18e3627843646dba"]
Oct 30 18:37:19.851128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.851107    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"]
Oct 30 18:37:19.854976 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.854932    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-wrong-role-denied-18e36275e41701f4"]
Oct 30 18:37:19.873507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.873478    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"]
Oct 30 18:37:19.877815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.877796    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-role-only-denied-18e362738598c36c"]
Oct 30 18:37:19.896623 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.896603    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"]
Oct 30 18:37:19.900862 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.900841    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-part-only-denied-18e36271273e1e1c"]
Oct 30 18:37:19.919771 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.919750    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"]
Oct 30 18:37:19.924117 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.924093    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-unlabeled-denied-18e3626ec8eba4cb"]
Oct 30 18:37:19.942519 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.942497    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"]
Oct 30 18:37:19.946695 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.946677    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-inferenceservice-allowed-18e3626c6a987ce6"]
Oct 30 18:37:19.964846 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.964822    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"]
Oct 30 18:37:19.968780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.968761    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-scheduler-allowed-18e3626a0c23c17e"]
Oct 30 18:37:19.987425 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.987406    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"]
Oct 30 18:37:19.993595 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:19.993568    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-prefill-allowed-18e36267adb9b610"]
Oct 30 18:37:20.007801 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.007776    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c801ba6-94b8-44ed-91d9-bf297e42396b" path="/var/lib/kubelet/pods/1c801ba6-94b8-44ed-91d9-bf297e42396b/volumes"
Oct 30 18:37:20.008106 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.008087    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21bbd240-586f-428f-a22b-d525c7f035fb" path="/var/lib/kubelet/pods/21bbd240-586f-428f-a22b-d525c7f035fb/volumes"
Oct 30 18:37:20.008344 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.008328    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24b1fa37-26d8-4d94-aca7-130db2eefa73" path="/var/lib/kubelet/pods/24b1fa37-26d8-4d94-aca7-130db2eefa73/volumes"
Oct 30 18:37:20.008584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.008568    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe" path="/var/lib/kubelet/pods/2f8a1b8c-088e-4d87-a2f4-9b380dc33cbe/volumes"
Oct 30 18:37:20.008809 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.008794    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39b64939-e788-4a14-9987-c246f6f89167" path="/var/lib/kubelet/pods/39b64939-e788-4a14-9987-c246f6f89167/volumes"
Oct 30 18:37:20.009379 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.009361    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5be0df53-5fd1-481e-9d00-5e8c022ee521" path="/var/lib/kubelet/pods/5be0df53-5fd1-481e-9d00-5e8c022ee521/volumes"
Oct 30 18:37:20.009624 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.009607    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65339edd-0e05-46b7-a7d8-55d759eaef39" path="/var/lib/kubelet/pods/65339edd-0e05-46b7-a7d8-55d759eaef39/volumes"
Oct 30 18:37:20.009886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.009871    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc" path="/var/lib/kubelet/pods/6c825a3c-97c7-48f7-85b7-7bce9f2aa9cc/volumes"
Oct 30 18:37:20.010166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.010149    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70897f9d-8556-4762-af73-21bb38f543f9" path="/var/lib/kubelet/pods/70897f9d-8556-4762-af73-21bb38f543f9/volumes"
Oct 30 18:37:20.010386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.010371    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c0c7390-ad9b-4fa7-b26d-dc869bc373fd" path="/var/lib/kubelet/pods/7c0c7390-ad9b-4fa7-b26d-dc869bc373fd/volumes"
Oct 30 18:37:20.010889 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.010871    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="806ea03c-ef8b-4cac-bc65-234ac96e8bc7" path="/var/lib/kubelet/pods/806ea03c-ef8b-4cac-bc65-234ac96e8bc7/volumes"
Oct 30 18:37:20.011154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.011137    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="892a0e88-c9f2-4b4d-8be5-046375e13828" path="/var/lib/kubelet/pods/892a0e88-c9f2-4b4d-8be5-046375e13828/volumes"
Oct 30 18:37:20.011374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.011358    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cc763e0-cae5-45ff-a1bf-f696b5c1dd62" path="/var/lib/kubelet/pods/8cc763e0-cae5-45ff-a1bf-f696b5c1dd62/volumes"
Oct 30 18:37:20.011600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.011585    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9019d2a3-dd51-46bf-adb3-f4180a465efe" path="/var/lib/kubelet/pods/9019d2a3-dd51-46bf-adb3-f4180a465efe/volumes"
Oct 30 18:37:20.011886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.011868    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ee527e8-b56a-49b6-b638-556fe61760e2" path="/var/lib/kubelet/pods/9ee527e8-b56a-49b6-b638-556fe61760e2/volumes"
Oct 30 18:37:20.012506 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.012487    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a40bb14a-083b-4efb-b95a-7d18fa10881f" path="/var/lib/kubelet/pods/a40bb14a-083b-4efb-b95a-7d18fa10881f/volumes"
Oct 30 18:37:20.012721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.012705    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5cb4d20-3159-454b-9a1c-3714fba15242" path="/var/lib/kubelet/pods/a5cb4d20-3159-454b-9a1c-3714fba15242/volumes"
Oct 30 18:37:20.012974 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.012958    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfb38fae-5220-409f-9dde-ada0d16ec40d" path="/var/lib/kubelet/pods/cfb38fae-5220-409f-9dde-ada0d16ec40d/volumes"
Oct 30 18:37:20.013193 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.013178    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10" path="/var/lib/kubelet/pods/d87b0f0b-c3e5-438a-9e8a-c591fa7e9c10/volumes"
Oct 30 18:37:20.013689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.013672    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e751edb8-b5b3-4e2f-911c-83b113b61899" path="/var/lib/kubelet/pods/e751edb8-b5b3-4e2f-911c-83b113b61899/volumes"
Oct 30 18:37:20.013938 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.013923    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4306185-d4af-4bd9-98da-40674725773c" path="/var/lib/kubelet/pods/f4306185-d4af-4bd9-98da-40674725773c/volumes"
Oct 30 18:37:20.014159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.014144    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f92286dc-91a4-4622-b15f-ffbf59d5008f" path="/var/lib/kubelet/pods/f92286dc-91a4-4622-b15f-ffbf59d5008f/volumes"
Oct 30 18:37:20.014369 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.014355    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc68792d-6dde-4398-a24d-eee8c143d22a" path="/var/lib/kubelet/pods/fc68792d-6dde-4398-a24d-eee8c143d22a/volumes"
Oct 30 18:37:20.014585 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.014570    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"]
Oct 30 18:37:20.016628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.016605    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-prefill-allowed-18e362654f57431a"]
Oct 30 18:37:20.035096 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.035078    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"]
Oct 30 18:37:20.039652 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.039634    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-leader-allowed-18e36262f0aa7fce"]
Oct 30 18:37:20.058372 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.058352    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"]
Oct 30 18:37:20.062426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.062407    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-worker-allowed-18e362609252702b"]
Oct 30 18:37:20.081567 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.081548    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"]
Oct 30 18:37:20.085634 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.085614    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-otlp-prefill-allowed-18e3625e33ebfd68"]
Oct 30 18:37:20.165329 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.165254    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"]
Oct 30 18:37:20.169590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:20.169563    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/collector-prometheus-allowed-18e3625786989977"]
Oct 30 18:37:21.285785 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.285748    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"]
Oct 30 18:37:21.286201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.286071    5746 cpu_manager.go:414] "containerMap: removing container" podUID="24b1fa37-26d8-4d94-aca7-130db2eefa73" containerName="curl"
Oct 30 18:37:21.286201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.286087    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24b1fa37-26d8-4d94-aca7-130db2eefa73" containerName="curl"
Oct 30 18:37:21.286201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.286151    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="24b1fa37-26d8-4d94-aca7-130db2eefa73" containerName="curl"
Oct 30 18:37:21.290457 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.290437    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"
Oct 30 18:37:21.307347 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.307320    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"]
Oct 30 18:37:21.318502 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.318484    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-namespace-auth-18e3628e5ac18847\"/\"default-dockercfg-4shnx\""
Oct 30 18:37:21.321571 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.321555    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"
Oct 30 18:37:21.615623 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:21.615590    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"]
Oct 30 18:37:21.618942 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:37:21.618902    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc845ba93_a760_4868_814c_984c974db595.slice/crio-16433cc62ba685a4b721986db29390ec8aee7958e90a52d89c7d225b80393eb3 WatchSource:0}: Error finding container 16433cc62ba685a4b721986db29390ec8aee7958e90a52d89c7d225b80393eb3: Status 404 returned error can't find the container with id 16433cc62ba685a4b721986db29390ec8aee7958e90a52d89c7d225b80393eb3
Oct 30 18:37:22.008404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:22.008326    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06ea13c7-83f8-43e5-8a1f-56c17cf29479" path="/var/lib/kubelet/pods/06ea13c7-83f8-43e5-8a1f-56c17cf29479/volumes"
Oct 30 18:37:22.008568 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:22.008556    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07fa27e4-5f8e-49ab-b902-b1b5f4304ebc" path="/var/lib/kubelet/pods/07fa27e4-5f8e-49ab-b902-b1b5f4304ebc/volumes"
Oct 30 18:37:22.008748 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:22.008737    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e113a893-9fb9-43c8-a1d5-ec05590e85a8" path="/var/lib/kubelet/pods/e113a893-9fb9-43c8-a1d5-ec05590e85a8/volumes"
Oct 30 18:37:22.008967 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:22.008956    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed3fe907-f3bb-4de0-bd25-02e1970585da" path="/var/lib/kubelet/pods/ed3fe907-f3bb-4de0-bd25-02e1970585da/volumes"
Oct 30 18:37:22.009145 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:22.009135    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f34a344f-a4cb-442e-b035-9fe2d43ebe80" path="/var/lib/kubelet/pods/f34a344f-a4cb-442e-b035-9fe2d43ebe80/volumes"
Oct 30 18:37:22.349966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:22.349932    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012" event={"ID":"c845ba93-a760-4868-814c-984c974db595","Type":"ContainerStarted","Data":"162d7414d639399e8950cf74b44a807487eeec789d7b7a1d0cd9bcbec3576a82"}
Oct 30 18:37:22.369569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:22.349977    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012" event={"ID":"c845ba93-a760-4868-814c-984c974db595","Type":"ContainerStarted","Data":"16433cc62ba685a4b721986db29390ec8aee7958e90a52d89c7d225b80393eb3"}
Oct 30 18:37:22.388006 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:22.387955    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012" podStartSLOduration=1.387943953 podStartE2EDuration="1.387943953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:22.387872734 +0000 UTC m=+2412.554690887" watchObservedRunningTime="2026-10-30 18:37:22.387943953 +0000 UTC m=+2412.554762074"
Oct 30 18:37:27.246173 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.246143    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"]
Oct 30 18:37:27.249380 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.249360    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.277927 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.277886    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"]
Oct 30 18:37:27.304813 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.304788    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8bl2\" (UniqueName: \"kubernetes.io/projected/10cd2032-ff7f-4e5c-afe3-d2f37e757524-kube-api-access-c8bl2\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.304976 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.304822    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/10cd2032-ff7f-4e5c-afe3-d2f37e757524-ta-internal\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.304976 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.304846    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/10cd2032-ff7f-4e5c-afe3-d2f37e757524-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.405639 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.405606    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/10cd2032-ff7f-4e5c-afe3-d2f37e757524-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.405819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.405683    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8bl2\" (UniqueName: \"kubernetes.io/projected/10cd2032-ff7f-4e5c-afe3-d2f37e757524-kube-api-access-c8bl2\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.405819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.405713    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/10cd2032-ff7f-4e5c-afe3-d2f37e757524-ta-internal\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.406456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.406435    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/10cd2032-ff7f-4e5c-afe3-d2f37e757524-ta-internal\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.408222 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.408202    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/10cd2032-ff7f-4e5c-afe3-d2f37e757524-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.428611 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.428586    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8bl2\" (UniqueName: \"kubernetes.io/projected/10cd2032-ff7f-4e5c-afe3-d2f37e757524-kube-api-access-c8bl2\") pod \"data-science-collector-targetallocator-77b99b589b-lcdcz\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.560757 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.560673    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:27.856270 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.856239    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"]
Oct 30 18:37:27.859279 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:37:27.859251    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10cd2032_ff7f_4e5c_afe3_d2f37e757524.slice/crio-beef1857fab60eac73e7e004e52e6ba00a7cd8cb7c591221a701f691721314ed WatchSource:0}: Error finding container beef1857fab60eac73e7e004e52e6ba00a7cd8cb7c591221a701f691721314ed: Status 404 returned error can't find the container with id beef1857fab60eac73e7e004e52e6ba00a7cd8cb7c591221a701f691721314ed
Oct 30 18:37:27.861447 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.861415    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:37:27.861531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:27.861474    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:37:28.386579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.386535    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz" event={"ID":"10cd2032-ff7f-4e5c-afe3-d2f37e757524","Type":"ContainerStarted","Data":"c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a"}
Oct 30 18:37:28.386579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.386586    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz" event={"ID":"10cd2032-ff7f-4e5c-afe3-d2f37e757524","Type":"ContainerStarted","Data":"beef1857fab60eac73e7e004e52e6ba00a7cd8cb7c591221a701f691721314ed"}
Oct 30 18:37:28.387128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.386607    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:28.388284 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.388268    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:28.429708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.429668    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz" podStartSLOduration=1.429654543 podStartE2EDuration="1.429654543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:28.429089836 +0000 UTC m=+2418.595907986" watchObservedRunningTime="2026-10-30 18:37:28.429654543 +0000 UTC m=+2418.596472692"
Oct 30 18:37:28.508125 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.508096    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"]
Oct 30 18:37:28.508302 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.508277    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5" podUID="41af1daf-dfb7-4c7c-b861-143aad7ba2a3" containerName="ta-container" containerID="cri-o://85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63" gracePeriod=30
Oct 30 18:37:28.740417 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.740393    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:37:28.815883 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.815848    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-ta-internal\" (UniqueName: \"kubernetes.io/configmap/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-ta-internal\") pod \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") "
Oct 30 18:37:28.816085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.815896    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-data-science-collector-ta-server-cert\") pod \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") "
Oct 30 18:37:28.816085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.815944    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-kube-api-access-26nch\" (UniqueName: \"kubernetes.io/projected/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-kube-api-access-26nch\") pod \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\" (UID: \"41af1daf-dfb7-4c7c-b861-143aad7ba2a3\") "
Oct 30 18:37:28.816252 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.816229    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-ta-internal" pod "41af1daf-dfb7-4c7c-b861-143aad7ba2a3" (UID: "41af1daf-dfb7-4c7c-b861-143aad7ba2a3"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:37:28.818100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.818080    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-data-science-collector-ta-server-cert" pod "41af1daf-dfb7-4c7c-b861-143aad7ba2a3" (UID: "41af1daf-dfb7-4c7c-b861-143aad7ba2a3"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:37:28.818187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.818164    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-kube-api-access-26nch" pod "41af1daf-dfb7-4c7c-b861-143aad7ba2a3" (UID: "41af1daf-dfb7-4c7c-b861-143aad7ba2a3"). InnerVolumeSpecName "kube-api-access-26nch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:28.917194 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.917164    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:28.917194 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.917194    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:28.917342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:28.917210    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26nch\" (UniqueName: \"kubernetes.io/projected/41af1daf-dfb7-4c7c-b861-143aad7ba2a3-kube-api-access-26nch\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:29.393177 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.393142    5746 generic.go:356] "Generic (PLEG): container finished" podID="41af1daf-dfb7-4c7c-b861-143aad7ba2a3" containerID="85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63" exitCode=0
Oct 30 18:37:29.393177 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.393179    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"
Oct 30 18:37:29.393753 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.393232    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5" event={"ID":"41af1daf-dfb7-4c7c-b861-143aad7ba2a3","Type":"ContainerDied","Data":"85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63"}
Oct 30 18:37:29.393753 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.393276    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5" event={"ID":"41af1daf-dfb7-4c7c-b861-143aad7ba2a3","Type":"ContainerDied","Data":"e276be7d53236aa5006040c4ecc447ab850e1566fc938ab0660a9a3dd5825e31"}
Oct 30 18:37:29.393753 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.393300    5746 scope.go:122] "RemoveContainer" containerID="85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63"
Oct 30 18:37:29.395016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.394992    5746 generic.go:356] "Generic (PLEG): container finished" podID="c845ba93-a760-4868-814c-984c974db595" containerID="162d7414d639399e8950cf74b44a807487eeec789d7b7a1d0cd9bcbec3576a82" exitCode=0
Oct 30 18:37:29.395130 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.395038    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012" event={"ID":"c845ba93-a760-4868-814c-984c974db595","Type":"ContainerDied","Data":"162d7414d639399e8950cf74b44a807487eeec789d7b7a1d0cd9bcbec3576a82"}
Oct 30 18:37:29.402408 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.402392    5746 scope.go:122] "RemoveContainer" containerID="85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63"
Oct 30 18:37:29.402697 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:29.402679    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63\": container with ID starting with 85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63 not found: ID does not exist" containerID="85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63"
Oct 30 18:37:29.402768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.402704    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63"} err="failed to get container status \"85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63\": rpc error: code = NotFound desc = could not find container \"85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63\": container with ID starting with 85046d0127ac94527466d8a737009d3ff58d60e3c9421f0429ca08e4cfd97a63 not found: ID does not exist"
Oct 30 18:37:29.480532 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.480466    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"]
Oct 30 18:37:29.484635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:29.484618    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-979d79b87-h2qh5"]
Oct 30 18:37:30.008306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:30.008264    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41af1daf-dfb7-4c7c-b861-143aad7ba2a3" path="/var/lib/kubelet/pods/41af1daf-dfb7-4c7c-b861-143aad7ba2a3/volumes"
Oct 30 18:37:30.527366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:30.527344    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"
Oct 30 18:37:31.406242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.406214    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"
Oct 30 18:37:31.406242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.406233    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012" event={"ID":"c845ba93-a760-4868-814c-984c974db595","Type":"ContainerDied","Data":"16433cc62ba685a4b721986db29390ec8aee7958e90a52d89c7d225b80393eb3"}
Oct 30 18:37:31.406471 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.406263    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16433cc62ba685a4b721986db29390ec8aee7958e90a52d89c7d225b80393eb3"
Oct 30 18:37:31.635635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.635605    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"]
Oct 30 18:37:31.636036 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.635853    5746 cpu_manager.go:414] "containerMap: removing container" podUID="c845ba93-a760-4868-814c-984c974db595" containerName="curl"
Oct 30 18:37:31.636036 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.635863    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c845ba93-a760-4868-814c-984c974db595" containerName="curl"
Oct 30 18:37:31.636036 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.635882    5746 cpu_manager.go:414] "containerMap: removing container" podUID="41af1daf-dfb7-4c7c-b861-143aad7ba2a3" containerName="ta-container"
Oct 30 18:37:31.636036 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.635886    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41af1daf-dfb7-4c7c-b861-143aad7ba2a3" containerName="ta-container"
Oct 30 18:37:31.636036 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.635958    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="41af1daf-dfb7-4c7c-b861-143aad7ba2a3" containerName="ta-container"
Oct 30 18:37:31.636036 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.635971    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="c845ba93-a760-4868-814c-984c974db595" containerName="curl"
Oct 30 18:37:31.638682 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.638665    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"
Oct 30 18:37:31.655814 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.655793    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"collector-namespace-auth-18e3628e5ac18847\"/\"default-dockercfg-4shnx\""
Oct 30 18:37:31.656050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.656031    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"]
Oct 30 18:37:31.660023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.659965    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"
Oct 30 18:37:31.953026 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:31.952949    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"]
Oct 30 18:37:31.956183 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:37:31.956159    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2150be49_f222_43a4_a116_029e349853e0.slice/crio-f99e9a893ae70746cbd5a2e02008d40b69dd87fd712f3e847e2d88631bf8c488 WatchSource:0}: Error finding container f99e9a893ae70746cbd5a2e02008d40b69dd87fd712f3e847e2d88631bf8c488: Status 404 returned error can't find the container with id f99e9a893ae70746cbd5a2e02008d40b69dd87fd712f3e847e2d88631bf8c488
Oct 30 18:37:32.412396 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:32.412364    5746 generic.go:356] "Generic (PLEG): container finished" podID="2150be49-f222-43a4-a116-029e349853e0" containerID="fd5b803f94b1a90981ab0eee01e81c1b70ee5b20a202e85283d7eaf4777aa2ff" exitCode=0
Oct 30 18:37:32.412563 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:32.412452    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9" event={"ID":"2150be49-f222-43a4-a116-029e349853e0","Type":"ContainerDied","Data":"fd5b803f94b1a90981ab0eee01e81c1b70ee5b20a202e85283d7eaf4777aa2ff"}
Oct 30 18:37:32.412563 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:32.412493    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9" event={"ID":"2150be49-f222-43a4-a116-029e349853e0","Type":"ContainerStarted","Data":"f99e9a893ae70746cbd5a2e02008d40b69dd87fd712f3e847e2d88631bf8c488"}
Oct 30 18:37:33.543286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:33.543263    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"
Oct 30 18:37:33.925353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:33.925320    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"]
Oct 30 18:37:33.925647 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:33.925633    5746 cpu_manager.go:414] "containerMap: removing container" podUID="2150be49-f222-43a4-a116-029e349853e0" containerName="curl"
Oct 30 18:37:33.925705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:33.925648    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2150be49-f222-43a4-a116-029e349853e0" containerName="curl"
Oct 30 18:37:33.925742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:33.925711    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="2150be49-f222-43a4-a116-029e349853e0" containerName="curl"
Oct 30 18:37:33.928458 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:33.928441    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"
Oct 30 18:37:33.939450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:33.939432    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"
Oct 30 18:37:33.946343 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:33.946319    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"]
Oct 30 18:37:34.233521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:34.233451    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"]
Oct 30 18:37:34.236801 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:37:34.236775    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod414d828a_1ded_410e_86e3_d91c2038461a.slice/crio-1dca67c108ad199878b4a2d3ead6ee1709322e8418151b46b65e2f9c84764249 WatchSource:0}: Error finding container 1dca67c108ad199878b4a2d3ead6ee1709322e8418151b46b65e2f9c84764249: Status 404 returned error can't find the container with id 1dca67c108ad199878b4a2d3ead6ee1709322e8418151b46b65e2f9c84764249
Oct 30 18:37:34.424037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:34.424013    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"
Oct 30 18:37:34.424199 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:34.424053    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9" event={"ID":"2150be49-f222-43a4-a116-029e349853e0","Type":"ContainerDied","Data":"f99e9a893ae70746cbd5a2e02008d40b69dd87fd712f3e847e2d88631bf8c488"}
Oct 30 18:37:34.424199 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:34.424093    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f99e9a893ae70746cbd5a2e02008d40b69dd87fd712f3e847e2d88631bf8c488"
Oct 30 18:37:34.425802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:34.425750    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506" event={"ID":"414d828a-1ded-410e-86e3-d91c2038461a","Type":"ContainerStarted","Data":"52e49117d7513a15e384f0175a0da6ed48399f9defdfed05aa124623fbbbfc71"}
Oct 30 18:37:34.425802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:34.425792    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506" event={"ID":"414d828a-1ded-410e-86e3-d91c2038461a","Type":"ContainerStarted","Data":"1dca67c108ad199878b4a2d3ead6ee1709322e8418151b46b65e2f9c84764249"}
Oct 30 18:37:34.463955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:34.463890    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506" podStartSLOduration=1.463879275 podStartE2EDuration="1.463879275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:34.463849221 +0000 UTC m=+2424.630667349" watchObservedRunningTime="2026-10-30 18:37:34.463879275 +0000 UTC m=+2424.630697402"
Oct 30 18:37:42.473617 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:42.473572    5746 generic.go:356] "Generic (PLEG): container finished" podID="414d828a-1ded-410e-86e3-d91c2038461a" containerID="52e49117d7513a15e384f0175a0da6ed48399f9defdfed05aa124623fbbbfc71" exitCode=0
Oct 30 18:37:42.474093 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:42.473644    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506" event={"ID":"414d828a-1ded-410e-86e3-d91c2038461a","Type":"ContainerDied","Data":"52e49117d7513a15e384f0175a0da6ed48399f9defdfed05aa124623fbbbfc71"}
Oct 30 18:37:42.790238 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:42.790159    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:37:43.609657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:43.609631    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"
Oct 30 18:37:44.027166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:44.027140    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"]
Oct 30 18:37:44.031075 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:44.031054    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"]
Oct 30 18:37:44.050118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:44.050096    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"]
Oct 30 18:37:44.053679 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:44.053660    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/enrolled-inference-allowed-18e3629101a5e9c9"]
Oct 30 18:37:44.072949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:44.072926    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"]
Oct 30 18:37:44.078711 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:44.078691    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["collector-namespace-auth-18e3628e5ac18847/unenrolled-spoof-denied-18e3628e97a73012"]
Oct 30 18:37:44.484383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:44.484354    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"
Oct 30 18:37:44.484383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:44.484387    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dca67c108ad199878b4a2d3ead6ee1709322e8418151b46b65e2f9c84764249"
Oct 30 18:37:44.499786 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:44.499752    5746 status_manager.go:1045] "Failed to get status for pod" err="pods \"revoked-inference-denied-18e362918a0a2506\" is forbidden: User \"system:node:ip-10-0-131-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"collector-namespace-auth-18e3628e5ac18847\": no relationship found between node 'ip-10-0-131-31.ec2.internal' and this object" podUID="414d828a-1ded-410e-86e3-d91c2038461a" pod="collector-namespace-auth-18e3628e5ac18847/revoked-inference-denied-18e362918a0a2506"
Oct 30 18:37:46.009606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:46.009550    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2150be49-f222-43a4-a116-029e349853e0" path="/var/lib/kubelet/pods/2150be49-f222-43a4-a116-029e349853e0/volumes"
Oct 30 18:37:46.010112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:46.009969    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="414d828a-1ded-410e-86e3-d91c2038461a" path="/var/lib/kubelet/pods/414d828a-1ded-410e-86e3-d91c2038461a/volumes"
Oct 30 18:37:46.010310 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:46.010290    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c845ba93-a760-4868-814c-984c974db595" path="/var/lib/kubelet/pods/c845ba93-a760-4868-814c-984c974db595/volumes"
Oct 30 18:37:50.624349 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.624321    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"]
Oct 30 18:37:50.624747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.624600    5746 cpu_manager.go:414] "containerMap: removing container" podUID="414d828a-1ded-410e-86e3-d91c2038461a" containerName="curl"
Oct 30 18:37:50.624747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.624611    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="414d828a-1ded-410e-86e3-d91c2038461a" containerName="curl"
Oct 30 18:37:50.624747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.624671    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="414d828a-1ded-410e-86e3-d91c2038461a" containerName="curl"
Oct 30 18:37:50.627790 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.627774    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.651733 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.651703    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"]
Oct 30 18:37:50.686944 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.686896    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.687055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.686952    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7z2r\" (UniqueName: \"kubernetes.io/projected/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-kube-api-access-j7z2r\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.687055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.687023    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.787504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.787481    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.787622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.787549    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.787622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.787573    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7z2r\" (UniqueName: \"kubernetes.io/projected/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-kube-api-access-j7z2r\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.788283 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.788257    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.790074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.790054    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.811234 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.811208    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7z2r\" (UniqueName: \"kubernetes.io/projected/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-kube-api-access-j7z2r\") pod \"data-science-collector-targetallocator-54cd4cbf97-2tfsw\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:50.938656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:50.938579    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:51.235298 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:51.235215    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"]
Oct 30 18:37:51.238564 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:37:51.238538    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60fd3d59_ac5e_4e4b_b2cc_eedcd4a7f7e7.slice/crio-2752bc30b952928acedfa218ae1bb1764736ef7e6152ed718fd5c50cd8d66769 WatchSource:0}: Error finding container 2752bc30b952928acedfa218ae1bb1764736ef7e6152ed718fd5c50cd8d66769: Status 404 returned error can't find the container with id 2752bc30b952928acedfa218ae1bb1764736ef7e6152ed718fd5c50cd8d66769
Oct 30 18:37:51.240609 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:51.240571    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:37:51.240707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:51.240633    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:37:51.524361 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:51.524265    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw" event={"ID":"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7","Type":"ContainerStarted","Data":"a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d"}
Oct 30 18:37:51.524361 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:51.524312    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw" event={"ID":"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7","Type":"ContainerStarted","Data":"2752bc30b952928acedfa218ae1bb1764736ef7e6152ed718fd5c50cd8d66769"}
Oct 30 18:37:51.524582 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:51.524407    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:51.525376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:51.525346    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw" podUID="60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.103:8080/readyz\": dial tcp 10.133.0.103:8080: connect: connection refused"
Oct 30 18:37:51.565593 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:51.565541    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw" podStartSLOduration=1.565530022 podStartE2EDuration="1.565530022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:51.565060875 +0000 UTC m=+2441.731879002" watchObservedRunningTime="2026-10-30 18:37:51.565530022 +0000 UTC m=+2441.732348143"
Oct 30 18:37:52.531419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.531393    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:52.608722 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.608688    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"]
Oct 30 18:37:52.608953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.608897    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz" podUID="10cd2032-ff7f-4e5c-afe3-d2f37e757524" containerName="ta-container" containerID="cri-o://c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a" gracePeriod=30
Oct 30 18:37:52.866039 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.866012    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:52.904048 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.904019    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/10cd2032-ff7f-4e5c-afe3-d2f37e757524-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/10cd2032-ff7f-4e5c-afe3-d2f37e757524-data-science-collector-ta-server-cert\") pod \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") "
Oct 30 18:37:52.904048 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.904065    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10cd2032-ff7f-4e5c-afe3-d2f37e757524-kube-api-access-c8bl2\" (UniqueName: \"kubernetes.io/projected/10cd2032-ff7f-4e5c-afe3-d2f37e757524-kube-api-access-c8bl2\") pod \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") "
Oct 30 18:37:52.904417 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.904158    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10cd2032-ff7f-4e5c-afe3-d2f37e757524-ta-internal\" (UniqueName: \"kubernetes.io/configmap/10cd2032-ff7f-4e5c-afe3-d2f37e757524-ta-internal\") pod \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\" (UID: \"10cd2032-ff7f-4e5c-afe3-d2f37e757524\") "
Oct 30 18:37:52.904715 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.904687    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10cd2032-ff7f-4e5c-afe3-d2f37e757524-ta-internal" pod "10cd2032-ff7f-4e5c-afe3-d2f37e757524" (UID: "10cd2032-ff7f-4e5c-afe3-d2f37e757524"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:37:52.908241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.908206    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10cd2032-ff7f-4e5c-afe3-d2f37e757524-data-science-collector-ta-server-cert" pod "10cd2032-ff7f-4e5c-afe3-d2f37e757524" (UID: "10cd2032-ff7f-4e5c-afe3-d2f37e757524"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:37:52.908381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.908208    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10cd2032-ff7f-4e5c-afe3-d2f37e757524-kube-api-access-c8bl2" pod "10cd2032-ff7f-4e5c-afe3-d2f37e757524" (UID: "10cd2032-ff7f-4e5c-afe3-d2f37e757524"). InnerVolumeSpecName "kube-api-access-c8bl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:52.912506 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.912477    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"]
Oct 30 18:37:52.958443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.958415    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:37:52.958659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:52.958623    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="874b8fa9-d753-4969-83ff-13ceba361446" containerName="perses" containerID="cri-o://19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891" gracePeriod=30
Oct 30 18:37:53.005239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.005215    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/10cd2032-ff7f-4e5c-afe3-d2f37e757524-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:53.005239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.005240    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/10cd2032-ff7f-4e5c-afe3-d2f37e757524-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:53.005386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.005250    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8bl2\" (UniqueName: \"kubernetes.io/projected/10cd2032-ff7f-4e5c-afe3-d2f37e757524-kube-api-access-c8bl2\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:53.191995 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.191972    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:53.308331 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.308297    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/874b8fa9-d753-4969-83ff-13ceba361446-kube-api-access-24njt\" (UniqueName: \"kubernetes.io/projected/874b8fa9-d753-4969-83ff-13ceba361446-kube-api-access-24njt\") pod \"874b8fa9-d753-4969-83ff-13ceba361446\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") "
Oct 30 18:37:53.308512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.308341    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/874b8fa9-d753-4969-83ff-13ceba361446-config\" (UniqueName: \"kubernetes.io/configmap/874b8fa9-d753-4969-83ff-13ceba361446-config\") pod \"874b8fa9-d753-4969-83ff-13ceba361446\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") "
Oct 30 18:37:53.308512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.308369    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/874b8fa9-d753-4969-83ff-13ceba361446-plugins\" (UniqueName: \"kubernetes.io/empty-dir/874b8fa9-d753-4969-83ff-13ceba361446-plugins\") pod \"874b8fa9-d753-4969-83ff-13ceba361446\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") "
Oct 30 18:37:53.308512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.308461    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"874b8fa9-d753-4969-83ff-13ceba361446\" (UID: \"874b8fa9-d753-4969-83ff-13ceba361446\") "
Oct 30 18:37:53.308768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.308736    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/874b8fa9-d753-4969-83ff-13ceba361446-config" pod "874b8fa9-d753-4969-83ff-13ceba361446" (UID: "874b8fa9-d753-4969-83ff-13ceba361446"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:37:53.310686 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.310660    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/874b8fa9-d753-4969-83ff-13ceba361446-kube-api-access-24njt" pod "874b8fa9-d753-4969-83ff-13ceba361446" (UID: "874b8fa9-d753-4969-83ff-13ceba361446"). InnerVolumeSpecName "kube-api-access-24njt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:53.311410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.311388    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "874b8fa9-d753-4969-83ff-13ceba361446" (UID: "874b8fa9-d753-4969-83ff-13ceba361446"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:37:53.409420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.409351    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24njt\" (UniqueName: \"kubernetes.io/projected/874b8fa9-d753-4969-83ff-13ceba361446-kube-api-access-24njt\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:53.409420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.409375    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/874b8fa9-d753-4969-83ff-13ceba361446-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:53.409420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.409408    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:37:53.422034 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.422010    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:37:53.451953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.451900    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874b8fa9-d753-4969-83ff-13ceba361446-plugins" pod "874b8fa9-d753-4969-83ff-13ceba361446" (UID: "874b8fa9-d753-4969-83ff-13ceba361446"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:37:53.510640 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.510615    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/874b8fa9-d753-4969-83ff-13ceba361446-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:53.510640 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.510637    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:37:53.536656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.536627    5746 generic.go:356] "Generic (PLEG): container finished" podID="874b8fa9-d753-4969-83ff-13ceba361446" containerID="19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891" exitCode=0
Oct 30 18:37:53.537020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.536678    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:53.537020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.536715    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"874b8fa9-d753-4969-83ff-13ceba361446","Type":"ContainerDied","Data":"19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891"}
Oct 30 18:37:53.537020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.536753    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"874b8fa9-d753-4969-83ff-13ceba361446","Type":"ContainerDied","Data":"0cccfaa938a93217edcd36607a61b851691a70c547a50e6ebbe03e769d75a940"}
Oct 30 18:37:53.537020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.536771    5746 scope.go:122] "RemoveContainer" containerID="19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891"
Oct 30 18:37:53.538411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.538295    5746 generic.go:356] "Generic (PLEG): container finished" podID="10cd2032-ff7f-4e5c-afe3-d2f37e757524" containerID="c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a" exitCode=0
Oct 30 18:37:53.538411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.538343    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"
Oct 30 18:37:53.538411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.538370    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz" event={"ID":"10cd2032-ff7f-4e5c-afe3-d2f37e757524","Type":"ContainerDied","Data":"c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a"}
Oct 30 18:37:53.538411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.538407    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz" event={"ID":"10cd2032-ff7f-4e5c-afe3-d2f37e757524","Type":"ContainerDied","Data":"beef1857fab60eac73e7e004e52e6ba00a7cd8cb7c591221a701f691721314ed"}
Oct 30 18:37:53.547531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.547515    5746 scope.go:122] "RemoveContainer" containerID="19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891"
Oct 30 18:37:53.547800 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:53.547781    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891\": container with ID starting with 19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891 not found: ID does not exist" containerID="19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891"
Oct 30 18:37:53.547867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.547812    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891"} err="failed to get container status \"19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891\": rpc error: code = NotFound desc = could not find container \"19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891\": container with ID starting with 19bc7c79805920d1cf193a7d3aa098643a6619db50718a05a4fba8c635dd6891 not found: ID does not exist"
Oct 30 18:37:53.547867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.547833    5746 scope.go:122] "RemoveContainer" containerID="c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a"
Oct 30 18:37:53.555965 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.555947    5746 scope.go:122] "RemoveContainer" containerID="c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a"
Oct 30 18:37:53.556200 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:53.556183    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a\": container with ID starting with c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a not found: ID does not exist" containerID="c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a"
Oct 30 18:37:53.556249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.556210    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a"} err="failed to get container status \"c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a\": rpc error: code = NotFound desc = could not find container \"c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a\": container with ID starting with c4eacf5a005358948c8ab2fba5e2d1886f2195b0a65b0f594b9d55ba3de6298a not found: ID does not exist"
Oct 30 18:37:53.587098 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.587073    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:37:53.591276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.591255    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:37:53.611260 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:53.611237    5746 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-collector-targetallocator: configmap "data-science-collector-targetallocator" not found
Oct 30 18:37:53.611375 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:53.611310    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal podName:60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:54.111289316 +0000 UTC m=+2444.278107435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ta-internal" (UniqueName: "kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal") pod "data-science-collector-targetallocator-54cd4cbf97-2tfsw" (UID: "60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7") : configmap "data-science-collector-targetallocator" not found
Oct 30 18:37:53.626017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.625997    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"]
Oct 30 18:37:53.628277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:53.628248    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-77b99b589b-lcdcz"]
Oct 30 18:37:54.008697 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.008665    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10cd2032-ff7f-4e5c-afe3-d2f37e757524" path="/var/lib/kubelet/pods/10cd2032-ff7f-4e5c-afe3-d2f37e757524/volumes"
Oct 30 18:37:54.009075 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.009061    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="874b8fa9-d753-4969-83ff-13ceba361446" path="/var/lib/kubelet/pods/874b8fa9-d753-4969-83ff-13ceba361446/volumes"
Oct 30 18:37:54.116560 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:54.116528    5746 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-collector-targetallocator: configmap "data-science-collector-targetallocator" not found
Oct 30 18:37:54.116760 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:54.116611    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal podName:60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:55.116596038 +0000 UTC m=+2445.283414147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ta-internal" (UniqueName: "kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal") pod "data-science-collector-targetallocator-54cd4cbf97-2tfsw" (UID: "60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7") : configmap "data-science-collector-targetallocator" not found
Oct 30 18:37:54.545595 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.545499    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw" podUID="60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" containerName="ta-container" containerID="cri-o://a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d" gracePeriod=30
Oct 30 18:37:54.786105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.786081    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:54.921972 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.921942    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-kube-api-access-j7z2r\" (UniqueName: \"kubernetes.io/projected/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-kube-api-access-j7z2r\") pod \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") "
Oct 30 18:37:54.922176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.922000    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal\" (UniqueName: \"kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal\") pod \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") "
Oct 30 18:37:54.922176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.922064    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-data-science-collector-ta-server-cert\") pod \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\" (UID: \"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7\") "
Oct 30 18:37:54.922351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.922318    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal" pod "60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" (UID: "60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:37:54.924223 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.924197    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-kube-api-access-j7z2r" pod "60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" (UID: "60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7"). InnerVolumeSpecName "kube-api-access-j7z2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:54.924326 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:54.924309    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-data-science-collector-ta-server-cert" pod "60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" (UID: "60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:37:55.023326 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.023299    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7z2r\" (UniqueName: \"kubernetes.io/projected/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-kube-api-access-j7z2r\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:55.023326 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.023323    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:55.023479 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.023334    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:37:55.551895 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.551862    5746 generic.go:356] "Generic (PLEG): container finished" podID="60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" containerID="a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d" exitCode=0
Oct 30 18:37:55.552405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.551921    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"
Oct 30 18:37:55.552405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.551951    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw" event={"ID":"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7","Type":"ContainerDied","Data":"a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d"}
Oct 30 18:37:55.552405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.552006    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw" event={"ID":"60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7","Type":"ContainerDied","Data":"2752bc30b952928acedfa218ae1bb1764736ef7e6152ed718fd5c50cd8d66769"}
Oct 30 18:37:55.552405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.552025    5746 scope.go:122] "RemoveContainer" containerID="a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d"
Oct 30 18:37:55.561404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.561387    5746 scope.go:122] "RemoveContainer" containerID="a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d"
Oct 30 18:37:55.561667 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:37:55.561649    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d\": container with ID starting with a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d not found: ID does not exist" containerID="a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d"
Oct 30 18:37:55.561726 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.561678    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d"} err="failed to get container status \"a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d\": rpc error: code = NotFound desc = could not find container \"a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d\": container with ID starting with a3372c00e5eb2b474d2f14cb7c689beceb3756cba44cb9cc213c80a8b9e31d3d not found: ID does not exist"
Oct 30 18:37:55.603172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.603142    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"]
Oct 30 18:37:55.615223 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:55.615194    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-54cd4cbf97-2tfsw"]
Oct 30 18:37:56.009494 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.009452    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" path="/var/lib/kubelet/pods/60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7/volumes"
Oct 30 18:37:56.639030 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639001    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639265    5746 cpu_manager.go:414] "containerMap: removing container" podUID="874b8fa9-d753-4969-83ff-13ceba361446" containerName="perses"
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639276    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="874b8fa9-d753-4969-83ff-13ceba361446" containerName="perses"
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639296    5746 cpu_manager.go:414] "containerMap: removing container" podUID="60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" containerName="ta-container"
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639301    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" containerName="ta-container"
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639312    5746 cpu_manager.go:414] "containerMap: removing container" podUID="10cd2032-ff7f-4e5c-afe3-d2f37e757524" containerName="ta-container"
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639317    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10cd2032-ff7f-4e5c-afe3-d2f37e757524" containerName="ta-container"
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639356    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="874b8fa9-d753-4969-83ff-13ceba361446" containerName="perses"
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639366    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="60fd3d59-ac5e-4e4b-b2cc-eedcd4a7f7e7" containerName="ta-container"
Oct 30 18:37:56.639422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.639373    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="10cd2032-ff7f-4e5c-afe3-d2f37e757524" containerName="ta-container"
Oct 30 18:37:56.643576 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.643557    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.662492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.662452    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:37:56.674417 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.674397    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:37:56.701489 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.701460    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:37:56.701635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.701519    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:37:56.701635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.701541    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:37:56.738764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.738737    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/1a04071e-1dba-420f-bda9-bcbefe4608b1-plugins\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.738874 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.738775    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a04071e-1dba-420f-bda9-bcbefe4608b1-config\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.738874 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.738858    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpljb\" (UniqueName: \"kubernetes.io/projected/1a04071e-1dba-420f-bda9-bcbefe4608b1-kube-api-access-mpljb\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.839757 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.839721    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/1a04071e-1dba-420f-bda9-bcbefe4608b1-plugins\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.839757 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.839759    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a04071e-1dba-420f-bda9-bcbefe4608b1-config\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.840008 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.839789    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpljb\" (UniqueName: \"kubernetes.io/projected/1a04071e-1dba-420f-bda9-bcbefe4608b1-kube-api-access-mpljb\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.840202 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.840179    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/1a04071e-1dba-420f-bda9-bcbefe4608b1-plugins\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.840464 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.840444    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a04071e-1dba-420f-bda9-bcbefe4608b1-config\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:37:56.862997 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:37:56.862969    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpljb\" (UniqueName: \"kubernetes.io/projected/1a04071e-1dba-420f-bda9-bcbefe4608b1-kube-api-access-mpljb\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:00.771007 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:00.770974    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:00.812588 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:00.812549    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:00.872015 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:00.871983    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:00.872153 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:00.872074    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:00.886348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:00.886323    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:00.989347 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:00.989311    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:01.018704 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:01.018680    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:01.155120 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:01.155088    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:01.454197 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:01.454006    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:38:01.456965 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:38:01.456935    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a04071e_1dba_420f_bda9_bcbefe4608b1.slice/crio-0c89b76b038b8cf920049aa6e6555b4c8a845e752d2a055a75f0528029366cf2 WatchSource:0}: Error finding container 0c89b76b038b8cf920049aa6e6555b4c8a845e752d2a055a75f0528029366cf2: Status 404 returned error can't find the container with id 0c89b76b038b8cf920049aa6e6555b4c8a845e752d2a055a75f0528029366cf2
Oct 30 18:38:01.587425 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:01.587391    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"1a04071e-1dba-420f-bda9-bcbefe4608b1","Type":"ContainerStarted","Data":"bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8"}
Oct 30 18:38:01.587628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:01.587438    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:01.587628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:01.587450    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"1a04071e-1dba-420f-bda9-bcbefe4608b1","Type":"ContainerStarted","Data":"0c89b76b038b8cf920049aa6e6555b4c8a845e752d2a055a75f0528029366cf2"}
Oct 30 18:38:01.628727 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:01.628625    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=5.628607956 podStartE2EDuration="5.628607956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:01.627550971 +0000 UTC m=+2451.794369097" watchObservedRunningTime="2026-10-30 18:38:01.628607956 +0000 UTC m=+2451.795426084"
Oct 30 18:38:12.595608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:12.595577    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:12.789825 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:38:12.789788    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:38:42.790596 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:38:42.790562    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:38:52.489470 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.489432    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:38:52.489986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.489670    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="1a04071e-1dba-420f-bda9-bcbefe4608b1" containerName="perses" containerID="cri-o://bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8" gracePeriod=30
Oct 30 18:38:52.594370 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.594328    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-perses-0" podUID="1a04071e-1dba-420f-bda9-bcbefe4608b1" containerName="perses" probeResult="failure" output="Get \"http://10.133.0.104:8080/metrics\": dial tcp 10.133.0.104:8080: connect: connection refused"
Oct 30 18:38:52.729674 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.729649    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:52.784023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.783950    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"1a04071e-1dba-420f-bda9-bcbefe4608b1\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") "
Oct 30 18:38:52.784023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.784000    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a04071e-1dba-420f-bda9-bcbefe4608b1-kube-api-access-mpljb\" (UniqueName: \"kubernetes.io/projected/1a04071e-1dba-420f-bda9-bcbefe4608b1-kube-api-access-mpljb\") pod \"1a04071e-1dba-420f-bda9-bcbefe4608b1\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") "
Oct 30 18:38:52.784208 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.784062    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a04071e-1dba-420f-bda9-bcbefe4608b1-config\" (UniqueName: \"kubernetes.io/configmap/1a04071e-1dba-420f-bda9-bcbefe4608b1-config\") pod \"1a04071e-1dba-420f-bda9-bcbefe4608b1\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") "
Oct 30 18:38:52.784208 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.784101    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a04071e-1dba-420f-bda9-bcbefe4608b1-plugins\" (UniqueName: \"kubernetes.io/empty-dir/1a04071e-1dba-420f-bda9-bcbefe4608b1-plugins\") pod \"1a04071e-1dba-420f-bda9-bcbefe4608b1\" (UID: \"1a04071e-1dba-420f-bda9-bcbefe4608b1\") "
Oct 30 18:38:52.784708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.784677    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a04071e-1dba-420f-bda9-bcbefe4608b1-config" pod "1a04071e-1dba-420f-bda9-bcbefe4608b1" (UID: "1a04071e-1dba-420f-bda9-bcbefe4608b1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:52.798404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.798372    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a04071e-1dba-420f-bda9-bcbefe4608b1-kube-api-access-mpljb" pod "1a04071e-1dba-420f-bda9-bcbefe4608b1" (UID: "1a04071e-1dba-420f-bda9-bcbefe4608b1"). InnerVolumeSpecName "kube-api-access-mpljb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:52.798874 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.798666    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "1a04071e-1dba-420f-bda9-bcbefe4608b1" (UID: "1a04071e-1dba-420f-bda9-bcbefe4608b1"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:38:52.831691 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.831647    5746 generic.go:356] "Generic (PLEG): container finished" podID="1a04071e-1dba-420f-bda9-bcbefe4608b1" containerID="bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8" exitCode=0
Oct 30 18:38:52.831838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.831715    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:52.831838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.831729    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"1a04071e-1dba-420f-bda9-bcbefe4608b1","Type":"ContainerDied","Data":"bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8"}
Oct 30 18:38:52.831838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.831783    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"1a04071e-1dba-420f-bda9-bcbefe4608b1","Type":"ContainerDied","Data":"0c89b76b038b8cf920049aa6e6555b4c8a845e752d2a055a75f0528029366cf2"}
Oct 30 18:38:52.831838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.831808    5746 scope.go:122] "RemoveContainer" containerID="bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8"
Oct 30 18:38:52.842560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.842526    5746 scope.go:122] "RemoveContainer" containerID="bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8"
Oct 30 18:38:52.842884 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:38:52.842862    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8\": container with ID starting with bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8 not found: ID does not exist" containerID="bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8"
Oct 30 18:38:52.843010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.842893    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8"} err="failed to get container status \"bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8\": rpc error: code = NotFound desc = could not find container \"bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8\": container with ID starting with bfff1ab5689c764dbd0f66b3528a1c44b4b25b54b05a2492512f86049b8c85f8 not found: ID does not exist"
Oct 30 18:38:52.885352 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.885322    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:38:52.885498 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.885356    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpljb\" (UniqueName: \"kubernetes.io/projected/1a04071e-1dba-420f-bda9-bcbefe4608b1-kube-api-access-mpljb\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:38:52.885498 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.885378    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a04071e-1dba-420f-bda9-bcbefe4608b1-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:38:52.897715 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.897695    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:38:52.946300 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.946266    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a04071e-1dba-420f-bda9-bcbefe4608b1-plugins" pod "1a04071e-1dba-420f-bda9-bcbefe4608b1" (UID: "1a04071e-1dba-420f-bda9-bcbefe4608b1"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:52.986340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.986315    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/1a04071e-1dba-420f-bda9-bcbefe4608b1-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:38:52.986340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:52.986336    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:38:53.174793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:53.174767    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:38:53.186733 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:53.186708    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:38:54.008132 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:54.008097    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a04071e-1dba-420f-bda9-bcbefe4608b1" path="/var/lib/kubelet/pods/1a04071e-1dba-420f-bda9-bcbefe4608b1/volumes"
Oct 30 18:38:56.388581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.388552    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:38:56.388981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.388816    5746 cpu_manager.go:414] "containerMap: removing container" podUID="1a04071e-1dba-420f-bda9-bcbefe4608b1" containerName="perses"
Oct 30 18:38:56.388981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.388826    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a04071e-1dba-420f-bda9-bcbefe4608b1" containerName="perses"
Oct 30 18:38:56.388981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.388886    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a04071e-1dba-420f-bda9-bcbefe4608b1" containerName="perses"
Oct 30 18:38:56.411393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.411369    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.415647 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.415621    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:38:56.459190 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.459164    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:38:56.459190 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.459164    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:38:56.463549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.463533    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:38:56.463609 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.463550    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:38:56.616662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.616635    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ed12f95-bf65-4842-b307-fb300b42b7b0-plugins\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.616831 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.616668    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ed12f95-bf65-4842-b307-fb300b42b7b0-config\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.616831 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.616697    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhkqb\" (UniqueName: \"kubernetes.io/projected/1ed12f95-bf65-4842-b307-fb300b42b7b0-kube-api-access-lhkqb\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.718081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.718008    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ed12f95-bf65-4842-b307-fb300b42b7b0-plugins\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.718081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.718038    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ed12f95-bf65-4842-b307-fb300b42b7b0-config\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.718081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.718069    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhkqb\" (UniqueName: \"kubernetes.io/projected/1ed12f95-bf65-4842-b307-fb300b42b7b0-kube-api-access-lhkqb\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.718418 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.718397    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ed12f95-bf65-4842-b307-fb300b42b7b0-plugins\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.718708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.718691    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ed12f95-bf65-4842-b307-fb300b42b7b0-config\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:38:56.742088 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:38:56.742061    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhkqb\" (UniqueName: \"kubernetes.io/projected/1ed12f95-bf65-4842-b307-fb300b42b7b0-kube-api-access-lhkqb\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:01.958986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:01.958952    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.000285 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.000255    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.060161 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.060118    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.060345 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.060250    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.074810 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.074784    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.178306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.178272    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.207694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.207665    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.423848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.423811    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.724563 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.724493    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:39:02.727886 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:39:02.727857    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ed12f95_bf65_4842_b307_fb300b42b7b0.slice/crio-9c11fb354346cf6ec063cd9ebbc816e668006e7858713326cd016ea8e29c3d06 WatchSource:0}: Error finding container 9c11fb354346cf6ec063cd9ebbc816e668006e7858713326cd016ea8e29c3d06: Status 404 returned error can't find the container with id 9c11fb354346cf6ec063cd9ebbc816e668006e7858713326cd016ea8e29c3d06
Oct 30 18:39:02.883043 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.883004    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"1ed12f95-bf65-4842-b307-fb300b42b7b0","Type":"ContainerStarted","Data":"cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97"}
Oct 30 18:39:02.883204 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.883060    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:02.883204 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.883079    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"1ed12f95-bf65-4842-b307-fb300b42b7b0","Type":"ContainerStarted","Data":"9c11fb354346cf6ec063cd9ebbc816e668006e7858713326cd016ea8e29c3d06"}
Oct 30 18:39:02.924425 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:02.924366    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=6.924351304 podStartE2EDuration="6.924351304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:02.923665254 +0000 UTC m=+2513.090483412" watchObservedRunningTime="2026-10-30 18:39:02.924351304 +0000 UTC m=+2513.091169430"
Oct 30 18:39:11.571276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.571238    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:39:11.571763 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.571480    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="1ed12f95-bf65-4842-b307-fb300b42b7b0" containerName="perses" containerID="cri-o://cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97" gracePeriod=30
Oct 30 18:39:11.827049 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.826994    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:11.929051 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.929019    5746 generic.go:356] "Generic (PLEG): container finished" podID="1ed12f95-bf65-4842-b307-fb300b42b7b0" containerID="cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97" exitCode=0
Oct 30 18:39:11.929209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.929071    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"1ed12f95-bf65-4842-b307-fb300b42b7b0","Type":"ContainerDied","Data":"cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97"}
Oct 30 18:39:11.929209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.929113    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"1ed12f95-bf65-4842-b307-fb300b42b7b0","Type":"ContainerDied","Data":"9c11fb354346cf6ec063cd9ebbc816e668006e7858713326cd016ea8e29c3d06"}
Oct 30 18:39:11.929209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.929133    5746 scope.go:122] "RemoveContainer" containerID="cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97"
Oct 30 18:39:11.929209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.929133    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:11.938651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.938630    5746 scope.go:122] "RemoveContainer" containerID="cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97"
Oct 30 18:39:11.939024 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:39:11.938978    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97\": container with ID starting with cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97 not found: ID does not exist" containerID="cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97"
Oct 30 18:39:11.939203 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.939031    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97"} err="failed to get container status \"cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97\": rpc error: code = NotFound desc = could not find container \"cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97\": container with ID starting with cd8a19dc625bc3d5781e44c59ffefba9a6db95197941bb222ee5f4ddf1ed0e97 not found: ID does not exist"
Oct 30 18:39:11.944338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.944314    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ed12f95-bf65-4842-b307-fb300b42b7b0-plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ed12f95-bf65-4842-b307-fb300b42b7b0-plugins\") pod \"1ed12f95-bf65-4842-b307-fb300b42b7b0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") "
Oct 30 18:39:11.944491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.944472    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"1ed12f95-bf65-4842-b307-fb300b42b7b0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") "
Oct 30 18:39:11.944581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.944523    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ed12f95-bf65-4842-b307-fb300b42b7b0-kube-api-access-lhkqb\" (UniqueName: \"kubernetes.io/projected/1ed12f95-bf65-4842-b307-fb300b42b7b0-kube-api-access-lhkqb\") pod \"1ed12f95-bf65-4842-b307-fb300b42b7b0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") "
Oct 30 18:39:11.944581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.944570    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ed12f95-bf65-4842-b307-fb300b42b7b0-config\" (UniqueName: \"kubernetes.io/configmap/1ed12f95-bf65-4842-b307-fb300b42b7b0-config\") pod \"1ed12f95-bf65-4842-b307-fb300b42b7b0\" (UID: \"1ed12f95-bf65-4842-b307-fb300b42b7b0\") "
Oct 30 18:39:11.945125 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.945102    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ed12f95-bf65-4842-b307-fb300b42b7b0-config" pod "1ed12f95-bf65-4842-b307-fb300b42b7b0" (UID: "1ed12f95-bf65-4842-b307-fb300b42b7b0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:39:11.949786 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.949759    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "1ed12f95-bf65-4842-b307-fb300b42b7b0" (UID: "1ed12f95-bf65-4842-b307-fb300b42b7b0"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:39:11.950059 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:11.950038    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ed12f95-bf65-4842-b307-fb300b42b7b0-kube-api-access-lhkqb" pod "1ed12f95-bf65-4842-b307-fb300b42b7b0" (UID: "1ed12f95-bf65-4842-b307-fb300b42b7b0"). InnerVolumeSpecName "kube-api-access-lhkqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:39:12.045822 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.045788    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:39:12.045998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.045835    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhkqb\" (UniqueName: \"kubernetes.io/projected/1ed12f95-bf65-4842-b307-fb300b42b7b0-kube-api-access-lhkqb\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:39:12.045998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.045848    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ed12f95-bf65-4842-b307-fb300b42b7b0-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:39:12.055285 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.055265    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:39:12.076581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.076555    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed12f95-bf65-4842-b307-fb300b42b7b0-plugins" pod "1ed12f95-bf65-4842-b307-fb300b42b7b0" (UID: "1ed12f95-bf65-4842-b307-fb300b42b7b0"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:39:12.146776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.146748    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:39:12.146776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.146773    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ed12f95-bf65-4842-b307-fb300b42b7b0-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:39:12.271700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.271673    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:39:12.274195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:12.274173    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:39:12.789948 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:39:12.789893    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:39:14.007810 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.007781    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ed12f95-bf65-4842-b307-fb300b42b7b0" path="/var/lib/kubelet/pods/1ed12f95-bf65-4842-b307-fb300b42b7b0/volumes"
Oct 30 18:39:14.762597 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.762565    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:39:14.762955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.762923    5746 cpu_manager.go:414] "containerMap: removing container" podUID="1ed12f95-bf65-4842-b307-fb300b42b7b0" containerName="perses"
Oct 30 18:39:14.763054 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.762957    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ed12f95-bf65-4842-b307-fb300b42b7b0" containerName="perses"
Oct 30 18:39:14.763124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.763115    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ed12f95-bf65-4842-b307-fb300b42b7b0" containerName="perses"
Oct 30 18:39:14.782838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.782810    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.793050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.793022    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:39:14.841931 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.841870    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:39:14.842109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.842027    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:39:14.842109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.842086    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:39:14.845590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.845566    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:39:14.868990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.868964    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nfvv\" (UniqueName: \"kubernetes.io/projected/db46e4ac-34d2-4929-8221-5d9154c08400-kube-api-access-4nfvv\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.869108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.869003    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db46e4ac-34d2-4929-8221-5d9154c08400-config\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.869108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.869025    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/db46e4ac-34d2-4929-8221-5d9154c08400-plugins\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.970009 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.969982    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nfvv\" (UniqueName: \"kubernetes.io/projected/db46e4ac-34d2-4929-8221-5d9154c08400-kube-api-access-4nfvv\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.970152 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.970028    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db46e4ac-34d2-4929-8221-5d9154c08400-config\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.970152 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.970050    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/db46e4ac-34d2-4929-8221-5d9154c08400-plugins\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.970391 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.970375    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/db46e4ac-34d2-4929-8221-5d9154c08400-plugins\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.970665 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.970647    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db46e4ac-34d2-4929-8221-5d9154c08400-config\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:14.993348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:14.993324    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nfvv\" (UniqueName: \"kubernetes.io/projected/db46e4ac-34d2-4929-8221-5d9154c08400-kube-api-access-4nfvv\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:22.429286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:22.429250    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:22.460596 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:22.460569    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:22.530140 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:22.530109    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:22.530288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:22.530242    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:22.544653 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:22.544633    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:22.648411 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:22.648375    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:22.677103 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:22.677077    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:22.895170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:22.895127    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:23.191567 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:23.191481    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:39:23.194443 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:39:23.194415    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb46e4ac_34d2_4929_8221_5d9154c08400.slice/crio-f9468c1c02d07e8cce8a9b2b35076856e89efbc0757ae5e9a120a3eb59ec0537 WatchSource:0}: Error finding container f9468c1c02d07e8cce8a9b2b35076856e89efbc0757ae5e9a120a3eb59ec0537: Status 404 returned error can't find the container with id f9468c1c02d07e8cce8a9b2b35076856e89efbc0757ae5e9a120a3eb59ec0537
Oct 30 18:39:23.996863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:23.996739    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"db46e4ac-34d2-4929-8221-5d9154c08400","Type":"ContainerStarted","Data":"899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6"}
Oct 30 18:39:23.996863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:23.996790    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"db46e4ac-34d2-4929-8221-5d9154c08400","Type":"ContainerStarted","Data":"f9468c1c02d07e8cce8a9b2b35076856e89efbc0757ae5e9a120a3eb59ec0537"}
Oct 30 18:39:23.996863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:23.996833    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:24.038433 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:24.038378    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=10.038363666 podStartE2EDuration="10.038363666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:24.037879479 +0000 UTC m=+2534.204697606" watchObservedRunningTime="2026-10-30 18:39:24.038363666 +0000 UTC m=+2534.205181795"
Oct 30 18:39:35.004192 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:39:35.004160    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:39:42.789387 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:39:42.789352    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:40:06.604310 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.604277    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:40:06.604948 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.604509    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="db46e4ac-34d2-4929-8221-5d9154c08400" containerName="perses" containerID="cri-o://899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6" gracePeriod=30
Oct 30 18:40:06.855550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.855487    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:06.950342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.950310    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db46e4ac-34d2-4929-8221-5d9154c08400-plugins\" (UniqueName: \"kubernetes.io/empty-dir/db46e4ac-34d2-4929-8221-5d9154c08400-plugins\") pod \"db46e4ac-34d2-4929-8221-5d9154c08400\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") "
Oct 30 18:40:06.950546 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.950439    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"db46e4ac-34d2-4929-8221-5d9154c08400\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") "
Oct 30 18:40:06.950546 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.950481    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db46e4ac-34d2-4929-8221-5d9154c08400-kube-api-access-4nfvv\" (UniqueName: \"kubernetes.io/projected/db46e4ac-34d2-4929-8221-5d9154c08400-kube-api-access-4nfvv\") pod \"db46e4ac-34d2-4929-8221-5d9154c08400\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") "
Oct 30 18:40:06.950546 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.950513    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db46e4ac-34d2-4929-8221-5d9154c08400-config\" (UniqueName: \"kubernetes.io/configmap/db46e4ac-34d2-4929-8221-5d9154c08400-config\") pod \"db46e4ac-34d2-4929-8221-5d9154c08400\" (UID: \"db46e4ac-34d2-4929-8221-5d9154c08400\") "
Oct 30 18:40:06.951154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.951080    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db46e4ac-34d2-4929-8221-5d9154c08400-config" pod "db46e4ac-34d2-4929-8221-5d9154c08400" (UID: "db46e4ac-34d2-4929-8221-5d9154c08400"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:40:06.955545 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.955518    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db46e4ac-34d2-4929-8221-5d9154c08400-kube-api-access-4nfvv" pod "db46e4ac-34d2-4929-8221-5d9154c08400" (UID: "db46e4ac-34d2-4929-8221-5d9154c08400"). InnerVolumeSpecName "kube-api-access-4nfvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:40:06.955843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:06.955823    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "db46e4ac-34d2-4929-8221-5d9154c08400" (UID: "db46e4ac-34d2-4929-8221-5d9154c08400"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:40:07.051080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.051052    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:40:07.051080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.051077    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nfvv\" (UniqueName: \"kubernetes.io/projected/db46e4ac-34d2-4929-8221-5d9154c08400-kube-api-access-4nfvv\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:40:07.051080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.051087    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db46e4ac-34d2-4929-8221-5d9154c08400-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:40:07.063162 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.063143    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:40:07.105970 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.105865    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db46e4ac-34d2-4929-8221-5d9154c08400-plugins" pod "db46e4ac-34d2-4929-8221-5d9154c08400" (UID: "db46e4ac-34d2-4929-8221-5d9154c08400"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:40:07.152104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.152071    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/db46e4ac-34d2-4929-8221-5d9154c08400-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:40:07.152104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.152100    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:40:07.197833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.197796    5746 generic.go:356] "Generic (PLEG): container finished" podID="db46e4ac-34d2-4929-8221-5d9154c08400" containerID="899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6" exitCode=0
Oct 30 18:40:07.197996 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.197845    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:07.197996 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.197884    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"db46e4ac-34d2-4929-8221-5d9154c08400","Type":"ContainerDied","Data":"899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6"}
Oct 30 18:40:07.197996 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.197944    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"db46e4ac-34d2-4929-8221-5d9154c08400","Type":"ContainerDied","Data":"f9468c1c02d07e8cce8a9b2b35076856e89efbc0757ae5e9a120a3eb59ec0537"}
Oct 30 18:40:07.197996 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.197963    5746 scope.go:122] "RemoveContainer" containerID="899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6"
Oct 30 18:40:07.207102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.207084    5746 scope.go:122] "RemoveContainer" containerID="899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6"
Oct 30 18:40:07.207378 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:40:07.207356    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6\": container with ID starting with 899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6 not found: ID does not exist" containerID="899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6"
Oct 30 18:40:07.207419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.207388    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6"} err="failed to get container status \"899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6\": rpc error: code = NotFound desc = could not find container \"899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6\": container with ID starting with 899771bc6ac18171676a85c4e9ab1c0f9c06daf9b772c90aa8646e8914be3ba6 not found: ID does not exist"
Oct 30 18:40:07.248771 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.248740    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:40:07.252817 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:07.252792    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:40:08.008393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:08.008316    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db46e4ac-34d2-4929-8221-5d9154c08400" path="/var/lib/kubelet/pods/db46e4ac-34d2-4929-8221-5d9154c08400/volumes"
Oct 30 18:40:10.158179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.158148    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:40:10.158658 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.158512    5746 cpu_manager.go:414] "containerMap: removing container" podUID="db46e4ac-34d2-4929-8221-5d9154c08400" containerName="perses"
Oct 30 18:40:10.158658 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.158528    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db46e4ac-34d2-4929-8221-5d9154c08400" containerName="perses"
Oct 30 18:40:10.158658 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.158624    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="db46e4ac-34d2-4929-8221-5d9154c08400" containerName="perses"
Oct 30 18:40:10.181466 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.181436    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:40:10.181624 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.181554    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:10.196321 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.196296    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:40:10.208334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.208315    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:40:10.208463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.208364    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:40:10.273727 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.273702    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nms48\" (UniqueName: \"kubernetes.io/projected/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-kube-api-access-nms48\") pod \"seaweedfs\" (UID: \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:10.273856 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.273739    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-data\") pod \"seaweedfs\" (UID: \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:10.374432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.374401    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-data\") pod \"seaweedfs\" (UID: \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:10.374557 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.374466    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nms48\" (UniqueName: \"kubernetes.io/projected/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-kube-api-access-nms48\") pod \"seaweedfs\" (UID: \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:10.374780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.374762    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-data\") pod \"seaweedfs\" (UID: \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:10.398477 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.398455    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nms48\" (UniqueName: \"kubernetes.io/projected/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-kube-api-access-nms48\") pod \"seaweedfs\" (UID: \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:10.480396 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.480339    5746 scope.go:122] "RemoveContainer" containerID="3a19b91b874c33d22c4a1ecd24bbc8ad71b025b0a33ed0c60c2b525253a88d18"
Oct 30 18:40:10.488920 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.488888    5746 scope.go:122] "RemoveContainer" containerID="3d521e75c7125e0905862b551f8b650a2b4d4f98cf7ebd5bf08fd8beaa41ecfd"
Oct 30 18:40:10.494894 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.494872    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:10.496669 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.496653    5746 scope.go:122] "RemoveContainer" containerID="d6cd46e515927df7abdea19fc6f48ea5ea501ff19696310d5ca72a8a309843bf"
Oct 30 18:40:10.794952 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.794846    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:40:10.797839 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:40:10.797813    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb2238a7_48ba_4d9b_9c37_0ddddeaa8034.slice/crio-d063228116c57b79d9b018456cac591628eda66d9ba335e117fe40a4c05bef1d WatchSource:0}: Error finding container d063228116c57b79d9b018456cac591628eda66d9ba335e117fe40a4c05bef1d: Status 404 returned error can't find the container with id d063228116c57b79d9b018456cac591628eda66d9ba335e117fe40a4c05bef1d
Oct 30 18:40:10.799675 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:10.799656    5746 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:40:11.219717 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:11.219688    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/seaweedfs" event={"ID":"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034","Type":"ContainerStarted","Data":"d063228116c57b79d9b018456cac591628eda66d9ba335e117fe40a4c05bef1d"}
Oct 30 18:40:12.798034 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:40:12.797996    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:40:16.249265 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:16.249229    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/seaweedfs" event={"ID":"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034","Type":"ContainerStarted","Data":"943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553"}
Oct 30 18:40:16.249707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:16.249340    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:16.289249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:16.289179    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/seaweedfs" podStartSLOduration=1.300669141 podStartE2EDuration="6.289162073s" totalImagesPullingTime="4.988492932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:16.288518873 +0000 UTC m=+2586.455337011" watchObservedRunningTime="2026-10-30 18:40:16.289162073 +0000 UTC m=+2586.455980203"
Oct 30 18:40:27.255636 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:27.255605    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:40:38.314882 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.314851    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:40:38.318368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.318347    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.340435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.340409    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:40:38.400781 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.400752    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:40:38.474056 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.474021    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-config\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.474056 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.474059    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vvfb\" (UniqueName: \"kubernetes.io/projected/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-kube-api-access-8vvfb\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.474265 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.474166    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-plugins\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.574856 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.574774    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-plugins\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.574856 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.574838    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-config\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.575124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.574859    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vvfb\" (UniqueName: \"kubernetes.io/projected/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-kube-api-access-8vvfb\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.575278 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.575255    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-plugins\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.575503 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.575484    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-config\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:38.598200 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:38.598178    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vvfb\" (UniqueName: \"kubernetes.io/projected/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-kube-api-access-8vvfb\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:42.790371 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:40:42.790335    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:40:44.119966 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.119931    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:44.148921 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.148889    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:44.220726 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.220696    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:44.220954 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.220822    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:44.235201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.235168    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:44.339139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.339109    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:44.370302 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.370233    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:44.629621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.629517    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:44.927835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:44.927755    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:40:44.931195 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:40:44.931154    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a2b786f_fe17_41a4_8760_1c4d2c6571ab.slice/crio-0f2a38fb2dac37d1068a5100fedb1e26751fa9a8f4321568fc7d4453281b8b13 WatchSource:0}: Error finding container 0f2a38fb2dac37d1068a5100fedb1e26751fa9a8f4321568fc7d4453281b8b13: Status 404 returned error can't find the container with id 0f2a38fb2dac37d1068a5100fedb1e26751fa9a8f4321568fc7d4453281b8b13
Oct 30 18:40:45.387212 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:45.387177    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"9a2b786f-fe17-41a4-8760-1c4d2c6571ab","Type":"ContainerStarted","Data":"66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62"}
Oct 30 18:40:45.387696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:45.387226    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"9a2b786f-fe17-41a4-8760-1c4d2c6571ab","Type":"ContainerStarted","Data":"0f2a38fb2dac37d1068a5100fedb1e26751fa9a8f4321568fc7d4453281b8b13"}
Oct 30 18:40:45.387696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:45.387273    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:40:45.430307 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:45.430258    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=7.430245723 podStartE2EDuration="7.430245723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:45.429169058 +0000 UTC m=+2615.595987272" watchObservedRunningTime="2026-10-30 18:40:45.430245723 +0000 UTC m=+2615.597063850"
Oct 30 18:40:56.394027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:40:56.393997    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:41:10.534616 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:10.534583    5746 scope.go:122] "RemoveContainer" containerID="460915081cbb634242c0e8e60cd3f7574901313b2d676cddd6440a62905ccc58"
Oct 30 18:41:10.543134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:10.543105    5746 scope.go:122] "RemoveContainer" containerID="cdd602568d8daa1ecb50f7d0172074b3ea0159c65d5e8539b8ec50eede7cbae1"
Oct 30 18:41:10.550556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:10.550540    5746 scope.go:122] "RemoveContainer" containerID="56b7cba45a6af504ab179bf491a552356cf1e125fc5f66edee41db97d6d7d0f5"
Oct 30 18:41:10.560410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:10.560391    5746 scope.go:122] "RemoveContainer" containerID="7d3aebed6696915b325ffe2152247ab6f7f1ea5cc218ec797d31c37c95bd2729"
Oct 30 18:41:10.567946 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:10.567930    5746 scope.go:122] "RemoveContainer" containerID="ddb68816ca5d5a50de1fb2ccae9c8548b1634fbd0735aac1e77b3b4604a2178c"
Oct 30 18:41:10.575425 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:10.575405    5746 scope.go:122] "RemoveContainer" containerID="a60349a53a92b0bb08caa466f7a24c8aa2f2ec9aa8c24df061ec0572ccfd77fe"
Oct 30 18:41:12.789713 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:41:12.789678    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:41:42.789796 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:41:42.789756    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:41:51.320064 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.320018    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:41:51.320536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.320306    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="9a2b786f-fe17-41a4-8760-1c4d2c6571ab" containerName="perses" containerID="cri-o://66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62" gracePeriod=30
Oct 30 18:41:51.555016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.554994    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:41:51.673357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.673325    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-plugins\") pod \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") "
Oct 30 18:41:51.673537 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.673448    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-config\" (UniqueName: \"kubernetes.io/configmap/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-config\") pod \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") "
Oct 30 18:41:51.673593 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.673571    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") "
Oct 30 18:41:51.673652 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.673619    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-kube-api-access-8vvfb\" (UniqueName: \"kubernetes.io/projected/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-kube-api-access-8vvfb\") pod \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\" (UID: \"9a2b786f-fe17-41a4-8760-1c4d2c6571ab\") "
Oct 30 18:41:51.673895 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.673864    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-config" pod "9a2b786f-fe17-41a4-8760-1c4d2c6571ab" (UID: "9a2b786f-fe17-41a4-8760-1c4d2c6571ab"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:41:51.679417 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.679362    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-kube-api-access-8vvfb" pod "9a2b786f-fe17-41a4-8760-1c4d2c6571ab" (UID: "9a2b786f-fe17-41a4-8760-1c4d2c6571ab"). InnerVolumeSpecName "kube-api-access-8vvfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:41:51.679930 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.679881    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "9a2b786f-fe17-41a4-8760-1c4d2c6571ab" (UID: "9a2b786f-fe17-41a4-8760-1c4d2c6571ab"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:41:51.691032 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.690999    5746 generic.go:356] "Generic (PLEG): container finished" podID="9a2b786f-fe17-41a4-8760-1c4d2c6571ab" containerID="66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62" exitCode=0
Oct 30 18:41:51.691147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.691062    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:41:51.691147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.691078    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"9a2b786f-fe17-41a4-8760-1c4d2c6571ab","Type":"ContainerDied","Data":"66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62"}
Oct 30 18:41:51.691147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.691127    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"9a2b786f-fe17-41a4-8760-1c4d2c6571ab","Type":"ContainerDied","Data":"0f2a38fb2dac37d1068a5100fedb1e26751fa9a8f4321568fc7d4453281b8b13"}
Oct 30 18:41:51.691319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.691148    5746 scope.go:122] "RemoveContainer" containerID="66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62"
Oct 30 18:41:51.702296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.702274    5746 scope.go:122] "RemoveContainer" containerID="66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62"
Oct 30 18:41:51.702657 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:41:51.702630    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62\": container with ID starting with 66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62 not found: ID does not exist" containerID="66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62"
Oct 30 18:41:51.702777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.702695    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62"} err="failed to get container status \"66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62\": rpc error: code = NotFound desc = could not find container \"66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62\": container with ID starting with 66de4da3d63481e02d640a0c7e959f02eed3e11b7ce6ab2ad4bb1cc42cce5b62 not found: ID does not exist"
Oct 30 18:41:51.774377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.774349    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:41:51.774562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.774395    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:41:51.774562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.774413    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vvfb\" (UniqueName: \"kubernetes.io/projected/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-kube-api-access-8vvfb\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:41:51.789606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.789553    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:41:51.836840 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.836795    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-plugins" pod "9a2b786f-fe17-41a4-8760-1c4d2c6571ab" (UID: "9a2b786f-fe17-41a4-8760-1c4d2c6571ab"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:41:51.875635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.875604    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:41:51.875635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:51.875632    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/9a2b786f-fe17-41a4-8760-1c4d2c6571ab-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:41:52.064076 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:52.064041    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:41:52.069503 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:52.069475    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:41:53.675930 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:53.675875    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:41:53.676410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:53.676124    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/seaweedfs" podUID="fb2238a7-48ba-4d9b-9c37-0ddddeaa8034" containerName="seaweedfs" containerID="cri-o://943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553" gracePeriod=30
Oct 30 18:41:54.007722 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:41:54.007643    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a2b786f-fe17-41a4-8760-1c4d2c6571ab" path="/var/lib/kubelet/pods/9a2b786f-fe17-41a4-8760-1c4d2c6571ab/volumes"
Oct 30 18:42:10.609675 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.609636    5746 scope.go:122] "RemoveContainer" containerID="6a9028373b2164f4d94d5712602ff64ed9a77cc58ba4cb443e5c2836f4dfa74e"
Oct 30 18:42:10.617919 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.617885    5746 scope.go:122] "RemoveContainer" containerID="e5671958b1509ab29d3648f2eeea27cf818a3b0a53cae4627c3ce22f9263d7a4"
Oct 30 18:42:10.625152 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.625135    5746 scope.go:122] "RemoveContainer" containerID="6d4826b0f1b669f9d0e4562ec7d6d2d23406d468f798c7cc0d961967c0935359"
Oct 30 18:42:10.632432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.632415    5746 scope.go:122] "RemoveContainer" containerID="cd6e0bbb15ef668fb249844dada08a6293eaaa666ccb86d59bc5125ee41ed218"
Oct 30 18:42:10.639320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.639299    5746 scope.go:122] "RemoveContainer" containerID="f46204425f7e2a4823216afcad0a16ca3740ed28a4d86e928c9f82afc57caa5a"
Oct 30 18:42:10.646379 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.646363    5746 scope.go:122] "RemoveContainer" containerID="5df9097955aa33c267a93bc4c93d7fe0bf24c1a3d039828f6e72ab56a67c388f"
Oct 30 18:42:10.653276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.653254    5746 scope.go:122] "RemoveContainer" containerID="f5b47f4c212adcb522b78902cab7cfa47977ea62ffbc830a81ac000c0edaa1ec"
Oct 30 18:42:10.660242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.660226    5746 scope.go:122] "RemoveContainer" containerID="808e0d625c6a8a52648b1dfbe6050b89c048aa8e24bd9e2e1d44d73b50b1ceac"
Oct 30 18:42:10.667127 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.667111    5746 scope.go:122] "RemoveContainer" containerID="b4dc892a82a4dfd7409d385a13156f118502d7e6eb3c9b9e46fa26fc88fbc1dc"
Oct 30 18:42:10.674256 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:10.674237    5746 scope.go:122] "RemoveContainer" containerID="a08ce1fcff8244700fccebe139ea641d7ced69374721836bf05835cec941f27a"
Oct 30 18:42:12.789547 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:42:12.789508    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:42:19.021852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.021821    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:42:19.200443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.200403    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-kube-api-access-nms48\" (UniqueName: \"kubernetes.io/projected/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-kube-api-access-nms48\") pod \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\" (UID: \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\") "
Oct 30 18:42:19.200663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.200535    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-data\" (UniqueName: \"kubernetes.io/empty-dir/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-data\") pod \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\" (UID: \"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034\") "
Oct 30 18:42:19.201740 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.201714    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-data" pod "fb2238a7-48ba-4d9b-9c37-0ddddeaa8034" (UID: "fb2238a7-48ba-4d9b-9c37-0ddddeaa8034"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:42:19.202735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.202710    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-kube-api-access-nms48" pod "fb2238a7-48ba-4d9b-9c37-0ddddeaa8034" (UID: "fb2238a7-48ba-4d9b-9c37-0ddddeaa8034"). InnerVolumeSpecName "kube-api-access-nms48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:42:19.301962 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.301930    5746 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-data\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:42:19.301962 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.301955    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nms48\" (UniqueName: \"kubernetes.io/projected/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034-kube-api-access-nms48\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:42:19.817198 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.817162    5746 generic.go:356] "Generic (PLEG): container finished" podID="fb2238a7-48ba-4d9b-9c37-0ddddeaa8034" containerID="943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553" exitCode=0
Oct 30 18:42:19.817408 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.817209    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:42:19.817408 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.817243    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/seaweedfs" event={"ID":"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034","Type":"ContainerDied","Data":"943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553"}
Oct 30 18:42:19.817408 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.817285    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/seaweedfs" event={"ID":"fb2238a7-48ba-4d9b-9c37-0ddddeaa8034","Type":"ContainerDied","Data":"d063228116c57b79d9b018456cac591628eda66d9ba335e117fe40a4c05bef1d"}
Oct 30 18:42:19.817408 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.817302    5746 scope.go:122] "RemoveContainer" containerID="943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553"
Oct 30 18:42:19.826496 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.826480    5746 scope.go:122] "RemoveContainer" containerID="943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553"
Oct 30 18:42:19.826759 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:42:19.826739    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553\": container with ID starting with 943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553 not found: ID does not exist" containerID="943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553"
Oct 30 18:42:19.826805 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.826768    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553"} err="failed to get container status \"943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553\": rpc error: code = NotFound desc = could not find container \"943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553\": container with ID starting with 943412feb06384da1621a4a839f21f018c3157ba04b2e8d8678db4e989b7c553 not found: ID does not exist"
Oct 30 18:42:19.867200 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.867175    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:42:19.872450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:19.872431    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:42:20.007577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:20.007542    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb2238a7-48ba-4d9b-9c37-0ddddeaa8034" path="/var/lib/kubelet/pods/fb2238a7-48ba-4d9b-9c37-0ddddeaa8034/volumes"
Oct 30 18:42:22.200663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.200628    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/fake-gcs-server"]
Oct 30 18:42:22.201072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.200894    5746 cpu_manager.go:414] "containerMap: removing container" podUID="fb2238a7-48ba-4d9b-9c37-0ddddeaa8034" containerName="seaweedfs"
Oct 30 18:42:22.201072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.200921    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb2238a7-48ba-4d9b-9c37-0ddddeaa8034" containerName="seaweedfs"
Oct 30 18:42:22.201072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.200932    5746 cpu_manager.go:414] "containerMap: removing container" podUID="9a2b786f-fe17-41a4-8760-1c4d2c6571ab" containerName="perses"
Oct 30 18:42:22.201072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.200937    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a2b786f-fe17-41a4-8760-1c4d2c6571ab" containerName="perses"
Oct 30 18:42:22.201072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.200991    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb2238a7-48ba-4d9b-9c37-0ddddeaa8034" containerName="seaweedfs"
Oct 30 18:42:22.201072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.201000    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a2b786f-fe17-41a4-8760-1c4d2c6571ab" containerName="perses"
Oct 30 18:42:22.205198 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.205180    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:22.222368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.222348    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:42:22.222580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.222550    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:42:22.234544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.234518    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:42:22.235245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.235229    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/fake-gcs-server"]
Oct 30 18:42:22.328952 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.328898    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwsmq\" (UniqueName: \"kubernetes.io/projected/d59d9411-ca51-4b51-8d82-ecbdb1915c75-kube-api-access-fwsmq\") pod \"fake-gcs-server\" (UID: \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\") " pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:22.329112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.329012    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d59d9411-ca51-4b51-8d82-ecbdb1915c75-data\") pod \"fake-gcs-server\" (UID: \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\") " pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:22.429578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.429477    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d59d9411-ca51-4b51-8d82-ecbdb1915c75-data\") pod \"fake-gcs-server\" (UID: \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\") " pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:22.429578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.429525    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwsmq\" (UniqueName: \"kubernetes.io/projected/d59d9411-ca51-4b51-8d82-ecbdb1915c75-kube-api-access-fwsmq\") pod \"fake-gcs-server\" (UID: \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\") " pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:22.439636 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.429882    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d59d9411-ca51-4b51-8d82-ecbdb1915c75-data\") pod \"fake-gcs-server\" (UID: \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\") " pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:22.453127 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.453075    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwsmq\" (UniqueName: \"kubernetes.io/projected/d59d9411-ca51-4b51-8d82-ecbdb1915c75-kube-api-access-fwsmq\") pod \"fake-gcs-server\" (UID: \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\") " pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:22.516559 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.516532    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:22.809479 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.809406    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/fake-gcs-server"]
Oct 30 18:42:22.813222 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:42:22.813197    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd59d9411_ca51_4b51_8d82_ecbdb1915c75.slice/crio-d400f4f5b7e307ec3de0a98570a7d23fe41aad589c81971b98f35a9151aa19a5 WatchSource:0}: Error finding container d400f4f5b7e307ec3de0a98570a7d23fe41aad589c81971b98f35a9151aa19a5: Status 404 returned error can't find the container with id d400f4f5b7e307ec3de0a98570a7d23fe41aad589c81971b98f35a9151aa19a5
Oct 30 18:42:22.831769 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:22.831740    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/fake-gcs-server" event={"ID":"d59d9411-ca51-4b51-8d82-ecbdb1915c75","Type":"ContainerStarted","Data":"d400f4f5b7e307ec3de0a98570a7d23fe41aad589c81971b98f35a9151aa19a5"}
Oct 30 18:42:24.841883 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:24.841840    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/fake-gcs-server" event={"ID":"d59d9411-ca51-4b51-8d82-ecbdb1915c75","Type":"ContainerStarted","Data":"5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695"}
Oct 30 18:42:24.842325 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:24.841938    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:24.881687 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:24.881642    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/fake-gcs-server" podStartSLOduration=1.8925401530000001 podStartE2EDuration="2.881628726s" totalImagesPullingTime="989.088573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:24.881080755 +0000 UTC m=+2715.047898883" watchObservedRunningTime="2026-10-30 18:42:24.881628726 +0000 UTC m=+2715.048446855"
Oct 30 18:42:30.848822 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:30.848792    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:42:38.267366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.267333    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:42:38.271321 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.271299    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.290485 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.290458    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:42:38.351305 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.351277    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:42:38.449718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.449676    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-config\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.449954 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.449740    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-plugins\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.449954 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.449816    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzvwn\" (UniqueName: \"kubernetes.io/projected/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-kube-api-access-vzvwn\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.550477 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.550393    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-plugins\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.550477 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.550433    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzvwn\" (UniqueName: \"kubernetes.io/projected/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-kube-api-access-vzvwn\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.550477 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.550467    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-config\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.550865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.550842    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-plugins\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.551117 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.551101    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-config\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:38.574062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:38.574041    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzvwn\" (UniqueName: \"kubernetes.io/projected/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-kube-api-access-vzvwn\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:42.789599 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:42:42.789566    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:42:46.612225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:46.612192    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:46.640834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:46.640803    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:46.712853 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:46.712824    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:46.713003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:46.712983    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:46.727504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:46.727481    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:46.833897 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:46.833867    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:46.863419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:46.863361    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:46.984005 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:46.983974    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:47.279683 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:47.279580    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:42:47.283163 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:42:47.283134    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda95e9d5a_1021_4a08_8bc7_9ce5b7aa214e.slice/crio-18c362fd82d540093419a9b92f74d64448705ce9c2fcbc9817a71bc75c5ccc51 WatchSource:0}: Error finding container 18c362fd82d540093419a9b92f74d64448705ce9c2fcbc9817a71bc75c5ccc51: Status 404 returned error can't find the container with id 18c362fd82d540093419a9b92f74d64448705ce9c2fcbc9817a71bc75c5ccc51
Oct 30 18:42:47.948014 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:47.947979    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e","Type":"ContainerStarted","Data":"54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938"}
Oct 30 18:42:47.948401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:47.948024    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e","Type":"ContainerStarted","Data":"18c362fd82d540093419a9b92f74d64448705ce9c2fcbc9817a71bc75c5ccc51"}
Oct 30 18:42:47.948401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:47.948112    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:42:47.989939 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:47.989872    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=9.989857515 podStartE2EDuration="9.989857515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:47.989339725 +0000 UTC m=+2738.156157856" watchObservedRunningTime="2026-10-30 18:42:47.989857515 +0000 UTC m=+2738.156675641"
Oct 30 18:42:58.956531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:42:58.956498    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:43:10.728331 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:43:10.728299    5746 scope.go:122] "RemoveContainer" containerID="1b3f9fc33a8be132a30517429931801ea42ed0ccb9d5f4e1b0ec7bf01c387769"
Oct 30 18:43:10.736531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:43:10.736506    5746 scope.go:122] "RemoveContainer" containerID="645e4c67eea22676e68897b1997eebcb45a997718ae1a7c327c4a729d11ad5e4"
Oct 30 18:43:10.744017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:43:10.743996    5746 scope.go:122] "RemoveContainer" containerID="24875b1351a2ee7475a00fa816225dd36de91392eb1b3cf8fc648d13b079f4b6"
Oct 30 18:43:10.751743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:43:10.751724    5746 scope.go:122] "RemoveContainer" containerID="bee778dad7f8ad4517d580f706c35405e988c44f80e5a540d6a68736e55e9111"
Oct 30 18:43:10.759737 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:43:10.759712    5746 scope.go:122] "RemoveContainer" containerID="e3f06717abab75a6591162b0df3615e2d7b433c73d4d71c801cc03d535ff1afb"
Oct 30 18:43:10.766848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:43:10.766833    5746 scope.go:122] "RemoveContainer" containerID="808288db1cb7fc9bc6cb6f66c8e72d2b26b7e59886b142c5826c5bdcb34a552e"
Oct 30 18:43:10.773731 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:43:10.773714    5746 scope.go:122] "RemoveContainer" containerID="fd22393f50a4fb89a49e8182453e6256f16c73f44a86b93d3aa61b479259c8f7"
Oct 30 18:43:10.780570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:43:10.780554    5746 scope.go:122] "RemoveContainer" containerID="221737772d1e3e03594a0405d3705b6500b2c5f2ea57a83bf570eac89cfb00de"
Oct 30 18:43:12.791179 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:43:12.791146    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:43:42.789190 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:43:42.789150    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:44:10.820128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:10.820096    5746 scope.go:122] "RemoveContainer" containerID="fd5b803f94b1a90981ab0eee01e81c1b70ee5b20a202e85283d7eaf4777aa2ff"
Oct 30 18:44:10.827943 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:10.827918    5746 scope.go:122] "RemoveContainer" containerID="162d7414d639399e8950cf74b44a807487eeec789d7b7a1d0cd9bcbec3576a82"
Oct 30 18:44:10.835052 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:10.835030    5746 scope.go:122] "RemoveContainer" containerID="52e49117d7513a15e384f0175a0da6ed48399f9defdfed05aa124623fbbbfc71"
Oct 30 18:44:10.841736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:10.841704    5746 scope.go:122] "RemoveContainer" containerID="f37eca64566601a36df7016070d61d616da35314be8e4cb14d4c5a7f4a8bc738"
Oct 30 18:44:12.789231 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:44:12.789193    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:44:20.624593 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.624556    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:20.625083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.624772    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" containerName="perses" containerID="cri-o://54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938" gracePeriod=30
Oct 30 18:44:20.883409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.883353    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:20.963575 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.963544    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") "
Oct 30 18:44:20.963783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.963636    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-plugins\" (UniqueName: \"kubernetes.io/empty-dir/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-plugins\") pod \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") "
Oct 30 18:44:20.963783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.963703    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-kube-api-access-vzvwn\" (UniqueName: \"kubernetes.io/projected/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-kube-api-access-vzvwn\") pod \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") "
Oct 30 18:44:20.963783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.963746    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-config\" (UniqueName: \"kubernetes.io/configmap/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-config\") pod \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\" (UID: \"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e\") "
Oct 30 18:44:20.964327 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.964282    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-config" pod "a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" (UID: "a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:44:20.967925 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.967883    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" (UID: "a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:44:20.969859 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:20.969829    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-kube-api-access-vzvwn" pod "a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" (UID: "a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e"). InnerVolumeSpecName "kube-api-access-vzvwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:21.064324 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.064290    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzvwn\" (UniqueName: \"kubernetes.io/projected/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-kube-api-access-vzvwn\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:21.064324 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.064321    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:21.064565 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.064353    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:44:21.079235 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.079209    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:44:21.129378 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.129340    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-plugins" pod "a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" (UID: "a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:44:21.165556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.165482    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:44:21.165556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.165510    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:21.316933 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.316876    5746 generic.go:356] "Generic (PLEG): container finished" podID="a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" containerID="54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938" exitCode=0
Oct 30 18:44:21.316933 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.316935    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e","Type":"ContainerDied","Data":"54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938"}
Oct 30 18:44:21.317188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.316954    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:21.317188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.316970    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e","Type":"ContainerDied","Data":"18c362fd82d540093419a9b92f74d64448705ce9c2fcbc9817a71bc75c5ccc51"}
Oct 30 18:44:21.317188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.316987    5746 scope.go:122] "RemoveContainer" containerID="54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938"
Oct 30 18:44:21.325734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.325715    5746 scope.go:122] "RemoveContainer" containerID="54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938"
Oct 30 18:44:21.326000 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:44:21.325981    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938\": container with ID starting with 54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938 not found: ID does not exist" containerID="54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938"
Oct 30 18:44:21.326075 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.326008    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938"} err="failed to get container status \"54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938\": rpc error: code = NotFound desc = could not find container \"54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938\": container with ID starting with 54c5c8393bcab81167a0e533002ee272fef10046f3f0dd54ce2bda9c96016938 not found: ID does not exist"
Oct 30 18:44:21.365493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.365467    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:21.370484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.370464    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:21.744662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.744629    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/fake-gcs-server"]
Oct 30 18:44:21.745151 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.744810    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/fake-gcs-server" podUID="d59d9411-ca51-4b51-8d82-ecbdb1915c75" containerName="fake-gcs-server" containerID="cri-o://5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695" gracePeriod=30
Oct 30 18:44:21.991041 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:21.991016    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:44:22.008456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.008427    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" path="/var/lib/kubelet/pods/a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e/volumes"
Oct 30 18:44:22.073241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.073212    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d59d9411-ca51-4b51-8d82-ecbdb1915c75-kube-api-access-fwsmq\" (UniqueName: \"kubernetes.io/projected/d59d9411-ca51-4b51-8d82-ecbdb1915c75-kube-api-access-fwsmq\") pod \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\" (UID: \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\") "
Oct 30 18:44:22.073241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.073252    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d59d9411-ca51-4b51-8d82-ecbdb1915c75-data\" (UniqueName: \"kubernetes.io/empty-dir/d59d9411-ca51-4b51-8d82-ecbdb1915c75-data\") pod \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\" (UID: \"d59d9411-ca51-4b51-8d82-ecbdb1915c75\") "
Oct 30 18:44:22.073606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.073580    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d59d9411-ca51-4b51-8d82-ecbdb1915c75-data" pod "d59d9411-ca51-4b51-8d82-ecbdb1915c75" (UID: "d59d9411-ca51-4b51-8d82-ecbdb1915c75"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:44:22.075463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.075439    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d59d9411-ca51-4b51-8d82-ecbdb1915c75-kube-api-access-fwsmq" pod "d59d9411-ca51-4b51-8d82-ecbdb1915c75" (UID: "d59d9411-ca51-4b51-8d82-ecbdb1915c75"). InnerVolumeSpecName "kube-api-access-fwsmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:22.174703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.174631    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwsmq\" (UniqueName: \"kubernetes.io/projected/d59d9411-ca51-4b51-8d82-ecbdb1915c75-kube-api-access-fwsmq\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:22.174703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.174658    5746 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d59d9411-ca51-4b51-8d82-ecbdb1915c75-data\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:22.321651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.321612    5746 generic.go:356] "Generic (PLEG): container finished" podID="d59d9411-ca51-4b51-8d82-ecbdb1915c75" containerID="5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695" exitCode=0
Oct 30 18:44:22.321815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.321657    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/fake-gcs-server"
Oct 30 18:44:22.321815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.321686    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/fake-gcs-server" event={"ID":"d59d9411-ca51-4b51-8d82-ecbdb1915c75","Type":"ContainerDied","Data":"5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695"}
Oct 30 18:44:22.321815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.321722    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/fake-gcs-server" event={"ID":"d59d9411-ca51-4b51-8d82-ecbdb1915c75","Type":"ContainerDied","Data":"d400f4f5b7e307ec3de0a98570a7d23fe41aad589c81971b98f35a9151aa19a5"}
Oct 30 18:44:22.321815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.321740    5746 scope.go:122] "RemoveContainer" containerID="5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695"
Oct 30 18:44:22.330144 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.330128    5746 scope.go:122] "RemoveContainer" containerID="5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695"
Oct 30 18:44:22.330388 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:44:22.330371    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695\": container with ID starting with 5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695 not found: ID does not exist" containerID="5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695"
Oct 30 18:44:22.330452 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.330398    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695"} err="failed to get container status \"5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695\": rpc error: code = NotFound desc = could not find container \"5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695\": container with ID starting with 5e12d48638796759ae0505421f8beb8fe6ec3615f9c3a6ccbb1e208963304695 not found: ID does not exist"
Oct 30 18:44:22.370258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.370231    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/fake-gcs-server"]
Oct 30 18:44:22.373746 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:22.373725    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/fake-gcs-server"]
Oct 30 18:44:24.008071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:24.008044    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d59d9411-ca51-4b51-8d82-ecbdb1915c75" path="/var/lib/kubelet/pods/d59d9411-ca51-4b51-8d82-ecbdb1915c75/volumes"
Oct 30 18:44:25.565995 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.565963    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:44:25.566416 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.566400    5746 cpu_manager.go:414] "containerMap: removing container" podUID="a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" containerName="perses"
Oct 30 18:44:25.566462 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.566420    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" containerName="perses"
Oct 30 18:44:25.566462 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.566449    5746 cpu_manager.go:414] "containerMap: removing container" podUID="d59d9411-ca51-4b51-8d82-ecbdb1915c75" containerName="fake-gcs-server"
Oct 30 18:44:25.566462 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.566458    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d59d9411-ca51-4b51-8d82-ecbdb1915c75" containerName="fake-gcs-server"
Oct 30 18:44:25.566549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.566535    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="d59d9411-ca51-4b51-8d82-ecbdb1915c75" containerName="fake-gcs-server"
Oct 30 18:44:25.566581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.566552    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="a95e9d5a-1021-4a08-8bc7-9ce5b7aa214e" containerName="perses"
Oct 30 18:44:25.571063 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.571042    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.580441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.580415    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:25.583964 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.583943    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.592136 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.592117    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:44:25.600932 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.600886    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a5ff538a-4789-47ed-84ee-546435c0ee65-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.601022 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.600995    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk2g9\" (UniqueName: \"kubernetes.io/projected/a5ff538a-4789-47ed-84ee-546435c0ee65-kube-api-access-kk2g9\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.601103 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.601087    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.601139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.601123    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.604835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.604816    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:25.634180 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.634154    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-7zmp2\""
Oct 30 18:44:25.634180 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.634177    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:44:25.634350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.634207    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:44:25.634350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.634222    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:44:25.659168 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.659141    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:44:25.659168 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.659148    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:44:25.659371 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.659314    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-af329f19\""
Oct 30 18:44:25.659371 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.659317    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:44:25.702186 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.702159    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.702316 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.702199    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a5ff538a-4789-47ed-84ee-546435c0ee65-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.702316 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.702219    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk2g9\" (UniqueName: \"kubernetes.io/projected/a5ff538a-4789-47ed-84ee-546435c0ee65-kube-api-access-kk2g9\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.702316 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.702245    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/dfb6efe2-f845-468a-b35e-017bc548ce84-plugins\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.702316 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.702277    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dfb6efe2-f845-468a-b35e-017bc548ce84-config\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.702316 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.702296    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fk5l\" (UniqueName: \"kubernetes.io/projected/dfb6efe2-f845-468a-b35e-017bc548ce84-kube-api-access-6fk5l\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.702559 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.702327    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.702858 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.702838    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a5ff538a-4789-47ed-84ee-546435c0ee65-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.704925 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.704890    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.705053 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.705035    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.726944 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.726924    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk2g9\" (UniqueName: \"kubernetes.io/projected/a5ff538a-4789-47ed-84ee-546435c0ee65-kube-api-access-kk2g9\") pod \"data-science-collector-collector-1\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:25.803043 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.803009    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dfb6efe2-f845-468a-b35e-017bc548ce84-config\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.803043 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.803045    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fk5l\" (UniqueName: \"kubernetes.io/projected/dfb6efe2-f845-468a-b35e-017bc548ce84-kube-api-access-6fk5l\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.803219 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.803106    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/dfb6efe2-f845-468a-b35e-017bc548ce84-plugins\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.803444 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.803430    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/dfb6efe2-f845-468a-b35e-017bc548ce84-plugins\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.803655 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.803634    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dfb6efe2-f845-468a-b35e-017bc548ce84-config\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.826319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.826270    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fk5l\" (UniqueName: \"kubernetes.io/projected/dfb6efe2-f845-468a-b35e-017bc548ce84-kube-api-access-6fk5l\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:25.885777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:25.885750    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:26.200755 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:26.200719    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:44:26.203686 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:44:26.203657    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ff538a_4789_47ed_84ee_546435c0ee65.slice/crio-24431d7863fd0a4a00ad99d6c5426b66c862891e91272da2940c1a26a3d57f15 WatchSource:0}: Error finding container 24431d7863fd0a4a00ad99d6c5426b66c862891e91272da2940c1a26a3d57f15: Status 404 returned error can't find the container with id 24431d7863fd0a4a00ad99d6c5426b66c862891e91272da2940c1a26a3d57f15
Oct 30 18:44:26.205692 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:26.205661    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:44:26.205751 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:26.205722    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:44:26.347745 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:26.347710    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"a5ff538a-4789-47ed-84ee-546435c0ee65","Type":"ContainerStarted","Data":"916fb9accd0a8136b492110bcd91af59e893a3780b842ba9a7cdef1f91ab6f89"}
Oct 30 18:44:26.347983 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:26.347768    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"a5ff538a-4789-47ed-84ee-546435c0ee65","Type":"ContainerStarted","Data":"24431d7863fd0a4a00ad99d6c5426b66c862891e91272da2940c1a26a3d57f15"}
Oct 30 18:44:26.389766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:26.389712    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-1" podStartSLOduration=1.389696539 podStartE2EDuration="1.389696539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:26.388723279 +0000 UTC m=+2836.555541428" watchObservedRunningTime="2026-10-30 18:44:26.389696539 +0000 UTC m=+2836.556514666"
Oct 30 18:44:28.524210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:28.524176    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:28.555853 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:28.555827    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:28.625366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:28.625336    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:28.625548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:28.625477    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:28.639819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:28.639796    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:28.740692 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:28.740662    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:28.772553 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:28.772528    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:28.895677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:28.895646    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:29.190497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:29.190420    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:29.193733 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:44:29.193701    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfb6efe2_f845_468a_b35e_017bc548ce84.slice/crio-01002afc34ecc57102c06d6302b5c665342cb22b23fc8ebf56be9f857c9f0b4e WatchSource:0}: Error finding container 01002afc34ecc57102c06d6302b5c665342cb22b23fc8ebf56be9f857c9f0b4e: Status 404 returned error can't find the container with id 01002afc34ecc57102c06d6302b5c665342cb22b23fc8ebf56be9f857c9f0b4e
Oct 30 18:44:29.361527 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:29.361488    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"dfb6efe2-f845-468a-b35e-017bc548ce84","Type":"ContainerStarted","Data":"4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220"}
Oct 30 18:44:29.361717 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:29.361536    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"dfb6efe2-f845-468a-b35e-017bc548ce84","Type":"ContainerStarted","Data":"01002afc34ecc57102c06d6302b5c665342cb22b23fc8ebf56be9f857c9f0b4e"}
Oct 30 18:44:29.361717 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:29.361669    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:29.401661 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:29.401486    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=4.401469292 podStartE2EDuration="4.401469292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:29.400990522 +0000 UTC m=+2839.567808649" watchObservedRunningTime="2026-10-30 18:44:29.401469292 +0000 UTC m=+2839.568287401"
Oct 30 18:44:40.369683 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:40.369648    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:42.790740 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:44:42.790703    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:44:43.230973 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.230939    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:44:43.231497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.231462    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-1" podUID="a5ff538a-4789-47ed-84ee-546435c0ee65" containerName="otc-container" containerID="cri-o://916fb9accd0a8136b492110bcd91af59e893a3780b842ba9a7cdef1f91ab6f89" gracePeriod=30
Oct 30 18:44:43.370342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.370306    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:43.370549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.370523    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="dfb6efe2-f845-468a-b35e-017bc548ce84" containerName="perses" containerID="cri-o://4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220" gracePeriod=30
Oct 30 18:44:43.415652 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.415621    5746 generic.go:356] "Generic (PLEG): container finished" podID="a5ff538a-4789-47ed-84ee-546435c0ee65" containerID="916fb9accd0a8136b492110bcd91af59e893a3780b842ba9a7cdef1f91ab6f89" exitCode=0
Oct 30 18:44:43.415798 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.415699    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"a5ff538a-4789-47ed-84ee-546435c0ee65","Type":"ContainerDied","Data":"916fb9accd0a8136b492110bcd91af59e893a3780b842ba9a7cdef1f91ab6f89"}
Oct 30 18:44:43.504443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.504419    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:43.548777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.548703    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5ff538a-4789-47ed-84ee-546435c0ee65-otc-internal\" (UniqueName: \"kubernetes.io/configmap/a5ff538a-4789-47ed-84ee-546435c0ee65-otc-internal\") pod \"a5ff538a-4789-47ed-84ee-546435c0ee65\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") "
Oct 30 18:44:43.548936 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.548792    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-otlp-tls-certs\") pod \"a5ff538a-4789-47ed-84ee-546435c0ee65\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") "
Oct 30 18:44:43.548936 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.548850    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-monitoring-tls-certs\") pod \"a5ff538a-4789-47ed-84ee-546435c0ee65\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") "
Oct 30 18:44:43.548936 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.548890    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5ff538a-4789-47ed-84ee-546435c0ee65-kube-api-access-kk2g9\" (UniqueName: \"kubernetes.io/projected/a5ff538a-4789-47ed-84ee-546435c0ee65-kube-api-access-kk2g9\") pod \"a5ff538a-4789-47ed-84ee-546435c0ee65\" (UID: \"a5ff538a-4789-47ed-84ee-546435c0ee65\") "
Oct 30 18:44:43.549861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.549828    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ff538a-4789-47ed-84ee-546435c0ee65-otc-internal" pod "a5ff538a-4789-47ed-84ee-546435c0ee65" (UID: "a5ff538a-4789-47ed-84ee-546435c0ee65"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:44:43.553225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.553202    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-otlp-tls-certs" pod "a5ff538a-4789-47ed-84ee-546435c0ee65" (UID: "a5ff538a-4789-47ed-84ee-546435c0ee65"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:44:43.555027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.555004    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-monitoring-tls-certs" pod "a5ff538a-4789-47ed-84ee-546435c0ee65" (UID: "a5ff538a-4789-47ed-84ee-546435c0ee65"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:44:43.555027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.555008    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ff538a-4789-47ed-84ee-546435c0ee65-kube-api-access-kk2g9" pod "a5ff538a-4789-47ed-84ee-546435c0ee65" (UID: "a5ff538a-4789-47ed-84ee-546435c0ee65"). InnerVolumeSpecName "kube-api-access-kk2g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:43.616609 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.616586    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.650139    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dfb6efe2-f845-468a-b35e-017bc548ce84-config\" (UniqueName: \"kubernetes.io/configmap/dfb6efe2-f845-468a-b35e-017bc548ce84-config\") pod \"dfb6efe2-f845-468a-b35e-017bc548ce84\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") "
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.650318    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"dfb6efe2-f845-468a-b35e-017bc548ce84\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") "
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.650366    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfb6efe2-f845-468a-b35e-017bc548ce84-kube-api-access-6fk5l\" (UniqueName: \"kubernetes.io/projected/dfb6efe2-f845-468a-b35e-017bc548ce84-kube-api-access-6fk5l\") pod \"dfb6efe2-f845-468a-b35e-017bc548ce84\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") "
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.650477    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfb6efe2-f845-468a-b35e-017bc548ce84-plugins\" (UniqueName: \"kubernetes.io/empty-dir/dfb6efe2-f845-468a-b35e-017bc548ce84-plugins\") pod \"dfb6efe2-f845-468a-b35e-017bc548ce84\" (UID: \"dfb6efe2-f845-468a-b35e-017bc548ce84\") "
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.650703    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.650722    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk2g9\" (UniqueName: \"kubernetes.io/projected/a5ff538a-4789-47ed-84ee-546435c0ee65-kube-api-access-kk2g9\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.650736    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/a5ff538a-4789-47ed-84ee-546435c0ee65-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.650747    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff538a-4789-47ed-84ee-546435c0ee65-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:43.652725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.651297    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfb6efe2-f845-468a-b35e-017bc548ce84-config" pod "dfb6efe2-f845-468a-b35e-017bc548ce84" (UID: "dfb6efe2-f845-468a-b35e-017bc548ce84"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:44:43.653770 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.653743    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb6efe2-f845-468a-b35e-017bc548ce84-kube-api-access-6fk5l" pod "dfb6efe2-f845-468a-b35e-017bc548ce84" (UID: "dfb6efe2-f845-468a-b35e-017bc548ce84"). InnerVolumeSpecName "kube-api-access-6fk5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:43.656446 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.656425    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "dfb6efe2-f845-468a-b35e-017bc548ce84" (UID: "dfb6efe2-f845-468a-b35e-017bc548ce84"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:44:43.751855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.751787    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dfb6efe2-f845-468a-b35e-017bc548ce84-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:43.751855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.751828    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:44:43.751855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.751839    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fk5l\" (UniqueName: \"kubernetes.io/projected/dfb6efe2-f845-468a-b35e-017bc548ce84-kube-api-access-6fk5l\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:43.764754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.764731    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:44:43.810358 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.810329    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb6efe2-f845-468a-b35e-017bc548ce84-plugins" pod "dfb6efe2-f845-468a-b35e-017bc548ce84" (UID: "dfb6efe2-f845-468a-b35e-017bc548ce84"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:44:43.853140 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.853109    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:44:43.853140 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:43.853136    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/dfb6efe2-f845-468a-b35e-017bc548ce84-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:44:44.421406 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.421369    5746 generic.go:356] "Generic (PLEG): container finished" podID="dfb6efe2-f845-468a-b35e-017bc548ce84" containerID="4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220" exitCode=0
Oct 30 18:44:44.421574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.421423    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:44.421574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.421465    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"dfb6efe2-f845-468a-b35e-017bc548ce84","Type":"ContainerDied","Data":"4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220"}
Oct 30 18:44:44.421574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.421513    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"dfb6efe2-f845-468a-b35e-017bc548ce84","Type":"ContainerDied","Data":"01002afc34ecc57102c06d6302b5c665342cb22b23fc8ebf56be9f857c9f0b4e"}
Oct 30 18:44:44.421574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.421537    5746 scope.go:122] "RemoveContainer" containerID="4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220"
Oct 30 18:44:44.423071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.422943    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"a5ff538a-4789-47ed-84ee-546435c0ee65","Type":"ContainerDied","Data":"24431d7863fd0a4a00ad99d6c5426b66c862891e91272da2940c1a26a3d57f15"}
Oct 30 18:44:44.423071 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.422967    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:44:44.429666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.429649    5746 scope.go:122] "RemoveContainer" containerID="4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220"
Oct 30 18:44:44.429970 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:44:44.429947    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220\": container with ID starting with 4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220 not found: ID does not exist" containerID="4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220"
Oct 30 18:44:44.430052 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.429980    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220"} err="failed to get container status \"4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220\": rpc error: code = NotFound desc = could not find container \"4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220\": container with ID starting with 4dd59c00d27e8185d1c24bac4a2a53e917b0fd2917d5cb4cc10d20d47117a220 not found: ID does not exist"
Oct 30 18:44:44.430052 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.429998    5746 scope.go:122] "RemoveContainer" containerID="916fb9accd0a8136b492110bcd91af59e893a3780b842ba9a7cdef1f91ab6f89"
Oct 30 18:44:44.471985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.471955    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:44:44.477650 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.477627    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:44:44.511492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.511463    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:44.515012 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:44.514987    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:46.008281 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:46.008247    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5ff538a-4789-47ed-84ee-546435c0ee65" path="/var/lib/kubelet/pods/a5ff538a-4789-47ed-84ee-546435c0ee65/volumes"
Oct 30 18:44:46.008665 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:46.008640    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfb6efe2-f845-468a-b35e-017bc548ce84" path="/var/lib/kubelet/pods/dfb6efe2-f845-468a-b35e-017bc548ce84/volumes"
Oct 30 18:44:52.982604 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:52.982570    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"]
Oct 30 18:44:52.983080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:52.982863    5746 cpu_manager.go:414] "containerMap: removing container" podUID="dfb6efe2-f845-468a-b35e-017bc548ce84" containerName="perses"
Oct 30 18:44:52.983080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:52.982875    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfb6efe2-f845-468a-b35e-017bc548ce84" containerName="perses"
Oct 30 18:44:52.983080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:52.982885    5746 cpu_manager.go:414] "containerMap: removing container" podUID="a5ff538a-4789-47ed-84ee-546435c0ee65" containerName="otc-container"
Oct 30 18:44:52.983080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:52.982890    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5ff538a-4789-47ed-84ee-546435c0ee65" containerName="otc-container"
Oct 30 18:44:52.983080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:52.982955    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfb6efe2-f845-468a-b35e-017bc548ce84" containerName="perses"
Oct 30 18:44:52.983080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:52.982964    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5ff538a-4789-47ed-84ee-546435c0ee65" containerName="otc-container"
Oct 30 18:44:52.985385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:52.985365    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"
Oct 30 18:44:53.003883 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.003855    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"]
Oct 30 18:44:53.012285 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.012263    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:44:53.012394 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.012294    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:44:53.012518 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.012503    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:44:53.026019 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.025998    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnx6b\" (UniqueName: \"kubernetes.io/projected/dc459ac3-473b-4170-827d-bcbbb3b246a5-kube-api-access-hnx6b\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm\" (UID: \"dc459ac3-473b-4170-827d-bcbbb3b246a5\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"
Oct 30 18:44:53.126993 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.126956    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnx6b\" (UniqueName: \"kubernetes.io/projected/dc459ac3-473b-4170-827d-bcbbb3b246a5-kube-api-access-hnx6b\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm\" (UID: \"dc459ac3-473b-4170-827d-bcbbb3b246a5\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"
Oct 30 18:44:53.150926 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.150877    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnx6b\" (UniqueName: \"kubernetes.io/projected/dc459ac3-473b-4170-827d-bcbbb3b246a5-kube-api-access-hnx6b\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm\" (UID: \"dc459ac3-473b-4170-827d-bcbbb3b246a5\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"
Oct 30 18:44:53.297134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.297059    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"
Oct 30 18:44:53.590401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.590324    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"]
Oct 30 18:44:53.593071 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:44:53.593043    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc459ac3_473b_4170_827d_bcbbb3b246a5.slice/crio-bf8ca607acad557986a34a109693c6c378a5eb748f4ec2de99b35a1b47142e37 WatchSource:0}: Error finding container bf8ca607acad557986a34a109693c6c378a5eb748f4ec2de99b35a1b47142e37: Status 404 returned error can't find the container with id bf8ca607acad557986a34a109693c6c378a5eb748f4ec2de99b35a1b47142e37
Oct 30 18:44:53.936137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.936102    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:53.939337 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.939319    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:53.958862 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:53.958840    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:54.017858 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.017830    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:44:54.028833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.028810    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"]
Oct 30 18:44:54.036178 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.036154    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9tmq\" (UniqueName: \"kubernetes.io/projected/101ee503-0f76-4a6b-af62-35f62eff638f-kube-api-access-v9tmq\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.036297 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.036197    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/101ee503-0f76-4a6b-af62-35f62eff638f-plugins\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.036340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.036288    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/101ee503-0f76-4a6b-af62-35f62eff638f-config\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.036429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.036414    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.051778 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.051753    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-klpwr\""
Oct 30 18:44:54.051875 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.051856    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:44:54.052891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.052871    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"]
Oct 30 18:44:54.118074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.118043    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:44:54.137451 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.137426    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpgvv\" (UniqueName: \"kubernetes.io/projected/06ab042d-6af1-47f0-8677-7473519b3e1d-kube-api-access-qpgvv\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.137554 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.137473    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9tmq\" (UniqueName: \"kubernetes.io/projected/101ee503-0f76-4a6b-af62-35f62eff638f-kube-api-access-v9tmq\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.137554 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.137494    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/06ab042d-6af1-47f0-8677-7473519b3e1d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.137671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.137621    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/101ee503-0f76-4a6b-af62-35f62eff638f-plugins\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.137728 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.137681    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/101ee503-0f76-4a6b-af62-35f62eff638f-config\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.137728 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.137715    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/06ab042d-6af1-47f0-8677-7473519b3e1d-ta-internal\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.138037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.138020    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/101ee503-0f76-4a6b-af62-35f62eff638f-plugins\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.138274 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.138258    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/101ee503-0f76-4a6b-af62-35f62eff638f-config\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.161633 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.161606    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9tmq\" (UniqueName: \"kubernetes.io/projected/101ee503-0f76-4a6b-af62-35f62eff638f-kube-api-access-v9tmq\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:54.238657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.238569    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgvv\" (UniqueName: \"kubernetes.io/projected/06ab042d-6af1-47f0-8677-7473519b3e1d-kube-api-access-qpgvv\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.238657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.238621    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/06ab042d-6af1-47f0-8677-7473519b3e1d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.238657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.238658    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/06ab042d-6af1-47f0-8677-7473519b3e1d-ta-internal\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.239450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.239429    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/06ab042d-6af1-47f0-8677-7473519b3e1d-ta-internal\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.241485 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.241460    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/06ab042d-6af1-47f0-8677-7473519b3e1d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.264981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.264953    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpgvv\" (UniqueName: \"kubernetes.io/projected/06ab042d-6af1-47f0-8677-7473519b3e1d-kube-api-access-qpgvv\") pod \"data-science-collector-targetallocator-84bf8f9c6-vntzb\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.347302 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.347275    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:54.469335 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.469307    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm" event={"ID":"dc459ac3-473b-4170-827d-bcbbb3b246a5","Type":"ContainerStarted","Data":"22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4"}
Oct 30 18:44:54.469474 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.469345    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm" event={"ID":"dc459ac3-473b-4170-827d-bcbbb3b246a5","Type":"ContainerStarted","Data":"bf8ca607acad557986a34a109693c6c378a5eb748f4ec2de99b35a1b47142e37"}
Oct 30 18:44:54.509011 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.508889    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm" podStartSLOduration=2.508875066 podStartE2EDuration="2.508875066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:54.508172445 +0000 UTC m=+2864.674990573" watchObservedRunningTime="2026-10-30 18:44:54.508875066 +0000 UTC m=+2864.675693189"
Oct 30 18:44:54.641491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.641459    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"]
Oct 30 18:44:54.644678 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:44:54.644646    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06ab042d_6af1_47f0_8677_7473519b3e1d.slice/crio-957003d9ed981d81c334af2cd5683a432545fac7db3f978c457972e768f38d8b WatchSource:0}: Error finding container 957003d9ed981d81c334af2cd5683a432545fac7db3f978c457972e768f38d8b: Status 404 returned error can't find the container with id 957003d9ed981d81c334af2cd5683a432545fac7db3f978c457972e768f38d8b
Oct 30 18:44:54.646726 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.646697    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:44:54.646795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:54.646752    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:44:55.474513 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:55.474480    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb" event={"ID":"06ab042d-6af1-47f0-8677-7473519b3e1d","Type":"ContainerStarted","Data":"72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d"}
Oct 30 18:44:55.474513 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:55.474525    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb" event={"ID":"06ab042d-6af1-47f0-8677-7473519b3e1d","Type":"ContainerStarted","Data":"957003d9ed981d81c334af2cd5683a432545fac7db3f978c457972e768f38d8b"}
Oct 30 18:44:55.475000 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:55.474650    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:55.476339 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:55.476323    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:44:55.516304 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:55.516255    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb" podStartSLOduration=1.516241506 podStartE2EDuration="1.516241506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:55.516156922 +0000 UTC m=+2865.682975048" watchObservedRunningTime="2026-10-30 18:44:55.516241506 +0000 UTC m=+2865.683059632"
Oct 30 18:44:58.976259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:58.976228    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:59.006821 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:59.006796    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:59.076956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:59.076925    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:59.077120 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:59.077069    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:59.091665 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:59.091645    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:59.197784 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:59.197753    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:59.227183 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:59.227129    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:59.350773 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:59.350738    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:44:59.646318 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:44:59.646284    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:44:59.649917 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:44:59.649878    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod101ee503_0f76_4a6b_af62_35f62eff638f.slice/crio-48db2fef86ae9568b8d93c81910a9cbf06611bb4e66b8c87543c5bbf08100581 WatchSource:0}: Error finding container 48db2fef86ae9568b8d93c81910a9cbf06611bb4e66b8c87543c5bbf08100581: Status 404 returned error can't find the container with id 48db2fef86ae9568b8d93c81910a9cbf06611bb4e66b8c87543c5bbf08100581
Oct 30 18:45:00.495755 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:00.495717    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"101ee503-0f76-4a6b-af62-35f62eff638f","Type":"ContainerStarted","Data":"3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87"}
Oct 30 18:45:00.495755 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:00.495756    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"101ee503-0f76-4a6b-af62-35f62eff638f","Type":"ContainerStarted","Data":"48db2fef86ae9568b8d93c81910a9cbf06611bb4e66b8c87543c5bbf08100581"}
Oct 30 18:45:00.496280 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:00.495835    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:45:00.539141 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:00.539074    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=7.539053567 podStartE2EDuration="7.539053567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:00.53807523 +0000 UTC m=+2870.704893359" watchObservedRunningTime="2026-10-30 18:45:00.539053567 +0000 UTC m=+2870.705871747"
Oct 30 18:45:02.058106 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.058070    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"]
Oct 30 18:45:02.058574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.058233    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm" podUID="dc459ac3-473b-4170-827d-bcbbb3b246a5" containerName="thanos-querier" containerID="cri-o://22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4" gracePeriod=30
Oct 30 18:45:02.300617 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.300592    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"
Oct 30 18:45:02.405721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.405682    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc459ac3-473b-4170-827d-bcbbb3b246a5-kube-api-access-hnx6b\" (UniqueName: \"kubernetes.io/projected/dc459ac3-473b-4170-827d-bcbbb3b246a5-kube-api-access-hnx6b\") pod \"dc459ac3-473b-4170-827d-bcbbb3b246a5\" (UID: \"dc459ac3-473b-4170-827d-bcbbb3b246a5\") "
Oct 30 18:45:02.407929 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.407872    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc459ac3-473b-4170-827d-bcbbb3b246a5-kube-api-access-hnx6b" pod "dc459ac3-473b-4170-827d-bcbbb3b246a5" (UID: "dc459ac3-473b-4170-827d-bcbbb3b246a5"). InnerVolumeSpecName "kube-api-access-hnx6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:02.504498 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.504459    5746 generic.go:356] "Generic (PLEG): container finished" podID="dc459ac3-473b-4170-827d-bcbbb3b246a5" containerID="22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4" exitCode=0
Oct 30 18:45:02.504694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.504520    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"
Oct 30 18:45:02.504694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.504567    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm" event={"ID":"dc459ac3-473b-4170-827d-bcbbb3b246a5","Type":"ContainerDied","Data":"22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4"}
Oct 30 18:45:02.504694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.504610    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm" event={"ID":"dc459ac3-473b-4170-827d-bcbbb3b246a5","Type":"ContainerDied","Data":"bf8ca607acad557986a34a109693c6c378a5eb748f4ec2de99b35a1b47142e37"}
Oct 30 18:45:02.504694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.504627    5746 scope.go:122] "RemoveContainer" containerID="22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4"
Oct 30 18:45:02.506283 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.506245    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnx6b\" (UniqueName: \"kubernetes.io/projected/dc459ac3-473b-4170-827d-bcbbb3b246a5-kube-api-access-hnx6b\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:45:02.512981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.512962    5746 scope.go:122] "RemoveContainer" containerID="22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4"
Oct 30 18:45:02.513273 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:45:02.513252    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4\": container with ID starting with 22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4 not found: ID does not exist" containerID="22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4"
Oct 30 18:45:02.513330 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.513282    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4"} err="failed to get container status \"22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4\": rpc error: code = NotFound desc = could not find container \"22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4\": container with ID starting with 22504ed667720394c9c74bda78c05a79c87552c308d95027e0eeba15befd3ab4 not found: ID does not exist"
Oct 30 18:45:02.554648 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.554620    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"]
Oct 30 18:45:02.558991 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:02.558963    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-9s5bm"]
Oct 30 18:45:04.007962 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:04.007928    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc459ac3-473b-4170-827d-bcbbb3b246a5" path="/var/lib/kubelet/pods/dc459ac3-473b-4170-827d-bcbbb3b246a5/volumes"
Oct 30 18:45:11.502428 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:11.502396    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:45:12.791740 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:45:12.791701    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:45:38.212879 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.212845    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"]
Oct 30 18:45:38.213362 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.213162    5746 cpu_manager.go:414] "containerMap: removing container" podUID="dc459ac3-473b-4170-827d-bcbbb3b246a5" containerName="thanos-querier"
Oct 30 18:45:38.213362 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.213176    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc459ac3-473b-4170-827d-bcbbb3b246a5" containerName="thanos-querier"
Oct 30 18:45:38.213362 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.213223    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc459ac3-473b-4170-827d-bcbbb3b246a5" containerName="thanos-querier"
Oct 30 18:45:38.216091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.216073    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.243401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.243374    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"]
Oct 30 18:45:38.295600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.295567    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqq98\" (UniqueName: \"kubernetes.io/projected/ad372e46-c110-44ab-9726-d8b24c10d4c6-kube-api-access-sqq98\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.295724 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.295608    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ad372e46-c110-44ab-9726-d8b24c10d4c6-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.295724 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.295712    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ad372e46-c110-44ab-9726-d8b24c10d4c6-ta-internal\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.396052 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.396018    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqq98\" (UniqueName: \"kubernetes.io/projected/ad372e46-c110-44ab-9726-d8b24c10d4c6-kube-api-access-sqq98\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.396185 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.396056    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ad372e46-c110-44ab-9726-d8b24c10d4c6-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.396185 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.396098    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ad372e46-c110-44ab-9726-d8b24c10d4c6-ta-internal\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.396653 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.396634    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ad372e46-c110-44ab-9726-d8b24c10d4c6-ta-internal\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.398757 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.398736    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ad372e46-c110-44ab-9726-d8b24c10d4c6-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.419632 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.419609    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqq98\" (UniqueName: \"kubernetes.io/projected/ad372e46-c110-44ab-9726-d8b24c10d4c6-kube-api-access-sqq98\") pod \"data-science-collector-targetallocator-658494f955-kdxjw\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.527267 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.527173    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:38.823123 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.821499    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"]
Oct 30 18:45:38.830634 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:45:38.830589    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad372e46_c110_44ab_9726_d8b24c10d4c6.slice/crio-669af8f8f5be38a947146b903d1578ea9d602cf0287d0e1fc91537f5056e51ea WatchSource:0}: Error finding container 669af8f8f5be38a947146b903d1578ea9d602cf0287d0e1fc91537f5056e51ea: Status 404 returned error can't find the container with id 669af8f8f5be38a947146b903d1578ea9d602cf0287d0e1fc91537f5056e51ea
Oct 30 18:45:38.833237 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.833194    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:45:38.833357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:38.833272    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:45:39.644215 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:39.644178    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw" event={"ID":"ad372e46-c110-44ab-9726-d8b24c10d4c6","Type":"ContainerStarted","Data":"67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd"}
Oct 30 18:45:39.644669 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:39.644222    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw" event={"ID":"ad372e46-c110-44ab-9726-d8b24c10d4c6","Type":"ContainerStarted","Data":"669af8f8f5be38a947146b903d1578ea9d602cf0287d0e1fc91537f5056e51ea"}
Oct 30 18:45:39.644669 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:39.644350    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:39.646029 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:39.646009    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:45:39.686175 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:39.686138    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw" podStartSLOduration=1.686126667 podStartE2EDuration="1.686126667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:39.685206008 +0000 UTC m=+2909.852024152" watchObservedRunningTime="2026-10-30 18:45:39.686126667 +0000 UTC m=+2909.852944793"
Oct 30 18:45:39.756117 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:39.756069    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"]
Oct 30 18:45:39.756307 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:39.756277    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb" podUID="06ab042d-6af1-47f0-8677-7473519b3e1d" containerName="ta-container" containerID="cri-o://72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d" gracePeriod=30
Oct 30 18:45:39.999540 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:39.999510    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:45:40.109749 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.109716    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06ab042d-6af1-47f0-8677-7473519b3e1d-kube-api-access-qpgvv\" (UniqueName: \"kubernetes.io/projected/06ab042d-6af1-47f0-8677-7473519b3e1d-kube-api-access-qpgvv\") pod \"06ab042d-6af1-47f0-8677-7473519b3e1d\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") "
Oct 30 18:45:40.109749 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.109759    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06ab042d-6af1-47f0-8677-7473519b3e1d-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/06ab042d-6af1-47f0-8677-7473519b3e1d-data-science-collector-ta-server-cert\") pod \"06ab042d-6af1-47f0-8677-7473519b3e1d\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") "
Oct 30 18:45:40.110042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.109794    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06ab042d-6af1-47f0-8677-7473519b3e1d-ta-internal\" (UniqueName: \"kubernetes.io/configmap/06ab042d-6af1-47f0-8677-7473519b3e1d-ta-internal\") pod \"06ab042d-6af1-47f0-8677-7473519b3e1d\" (UID: \"06ab042d-6af1-47f0-8677-7473519b3e1d\") "
Oct 30 18:45:40.110261 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.110226    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06ab042d-6af1-47f0-8677-7473519b3e1d-ta-internal" pod "06ab042d-6af1-47f0-8677-7473519b3e1d" (UID: "06ab042d-6af1-47f0-8677-7473519b3e1d"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:40.112166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.112139    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06ab042d-6af1-47f0-8677-7473519b3e1d-data-science-collector-ta-server-cert" pod "06ab042d-6af1-47f0-8677-7473519b3e1d" (UID: "06ab042d-6af1-47f0-8677-7473519b3e1d"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:45:40.112259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.112205    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ab042d-6af1-47f0-8677-7473519b3e1d-kube-api-access-qpgvv" pod "06ab042d-6af1-47f0-8677-7473519b3e1d" (UID: "06ab042d-6af1-47f0-8677-7473519b3e1d"). InnerVolumeSpecName "kube-api-access-qpgvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:40.210788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.210695    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/06ab042d-6af1-47f0-8677-7473519b3e1d-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:45:40.210788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.210737    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpgvv\" (UniqueName: \"kubernetes.io/projected/06ab042d-6af1-47f0-8677-7473519b3e1d-kube-api-access-qpgvv\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:45:40.210788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.210754    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/06ab042d-6af1-47f0-8677-7473519b3e1d-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:45:40.649497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.649462    5746 generic.go:356] "Generic (PLEG): container finished" podID="06ab042d-6af1-47f0-8677-7473519b3e1d" containerID="72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d" exitCode=0
Oct 30 18:45:40.650016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.649512    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"
Oct 30 18:45:40.650016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.649545    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb" event={"ID":"06ab042d-6af1-47f0-8677-7473519b3e1d","Type":"ContainerDied","Data":"72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d"}
Oct 30 18:45:40.650016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.649595    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb" event={"ID":"06ab042d-6af1-47f0-8677-7473519b3e1d","Type":"ContainerDied","Data":"957003d9ed981d81c334af2cd5683a432545fac7db3f978c457972e768f38d8b"}
Oct 30 18:45:40.650016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.649613    5746 scope.go:122] "RemoveContainer" containerID="72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d"
Oct 30 18:45:40.657869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.657853    5746 scope.go:122] "RemoveContainer" containerID="72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d"
Oct 30 18:45:40.658143 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:45:40.658125    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d\": container with ID starting with 72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d not found: ID does not exist" containerID="72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d"
Oct 30 18:45:40.658212 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.658154    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d"} err="failed to get container status \"72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d\": rpc error: code = NotFound desc = could not find container \"72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d\": container with ID starting with 72d86a06b66206c62bd019fdf51e8385403f12cfafef239d0f3cbf4566b7356d not found: ID does not exist"
Oct 30 18:45:40.699805 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.699779    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"]
Oct 30 18:45:40.703481 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.703463    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-84bf8f9c6-vntzb"]
Oct 30 18:45:40.993368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.993293    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:45:40.993601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.993588    5746 cpu_manager.go:414] "containerMap: removing container" podUID="06ab042d-6af1-47f0-8677-7473519b3e1d" containerName="ta-container"
Oct 30 18:45:40.993646 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.993602    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06ab042d-6af1-47f0-8677-7473519b3e1d" containerName="ta-container"
Oct 30 18:45:40.993682 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.993649    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="06ab042d-6af1-47f0-8677-7473519b3e1d" containerName="ta-container"
Oct 30 18:45:40.997878 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:40.997858    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.024787 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.024749    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:45:41.024985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.024860    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:45:41.024985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.024879    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsglb\""
Oct 30 18:45:41.024985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.024887    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:45:41.024985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.024886    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-a0f9182a\""
Oct 30 18:45:41.025226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.025080    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:45:41.025767 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.025744    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:45:41.117615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.117578    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbmpz\" (UniqueName: \"kubernetes.io/projected/e8fb799e-99c0-41b4-885c-00946a448ea5-kube-api-access-qbmpz\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.117615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.117623    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.117833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.117689    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.117833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.117746    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.117833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.117802    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.117948 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.117837    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8fb799e-99c0-41b4-885c-00946a448ea5-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.218028 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.217995    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"]
Oct 30 18:45:41.218330 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.218309    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.218374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.218346    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.218409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.218376    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8fb799e-99c0-41b4-885c-00946a448ea5-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.218441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.218430    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbmpz\" (UniqueName: \"kubernetes.io/projected/e8fb799e-99c0-41b4-885c-00946a448ea5-kube-api-access-qbmpz\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.218543 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.218475    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.218543 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.218510    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.219450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.219424    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8fb799e-99c0-41b4-885c-00946a448ea5-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.221264 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.221239    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.221374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.221268    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.221374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.221353    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.221448 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.221424    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.227353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.227334    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"
Oct 30 18:45:41.238287 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.238267    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"]
Oct 30 18:45:41.244703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.244653    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"
Oct 30 18:45:41.245188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.245171    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbmpz\" (UniqueName: \"kubernetes.io/projected/e8fb799e-99c0-41b4-885c-00946a448ea5-kube-api-access-qbmpz\") pod \"data-science-collector-collector-1\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.309763 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.309734    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:45:41.538664 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.538575    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"]
Oct 30 18:45:41.541945 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:45:41.541898    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6c00844_e44d_4541_98db_538abac52cb2.slice/crio-d852575caec07fb2deb658688a817553e7630d7290b522a04cc78f45bc54b11e WatchSource:0}: Error finding container d852575caec07fb2deb658688a817553e7630d7290b522a04cc78f45bc54b11e: Status 404 returned error can't find the container with id d852575caec07fb2deb658688a817553e7630d7290b522a04cc78f45bc54b11e
Oct 30 18:45:41.612575 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.612545    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:45:41.615680 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:45:41.615651    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fb799e_99c0_41b4_885c_00946a448ea5.slice/crio-6114c97dcafb12bcb7d38926c31608ce793cc6c35400aa4fd1f80c31316781e5 WatchSource:0}: Error finding container 6114c97dcafb12bcb7d38926c31608ce793cc6c35400aa4fd1f80c31316781e5: Status 404 returned error can't find the container with id 6114c97dcafb12bcb7d38926c31608ce793cc6c35400aa4fd1f80c31316781e5
Oct 30 18:45:41.618159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.618131    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:45:41.618241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.618194    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:45:41.655803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.655774    5746 generic.go:356] "Generic (PLEG): container finished" podID="b6c00844-e44d-4541-98db-538abac52cb2" containerID="6c549558731430279cb0ae8a785e9cb488a3d4ac9366c8163b5604933972a7e6" exitCode=0
Oct 30 18:45:41.656233 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.655831    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5" event={"ID":"b6c00844-e44d-4541-98db-538abac52cb2","Type":"ContainerDied","Data":"6c549558731430279cb0ae8a785e9cb488a3d4ac9366c8163b5604933972a7e6"}
Oct 30 18:45:41.656233 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.655873    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5" event={"ID":"b6c00844-e44d-4541-98db-538abac52cb2","Type":"ContainerStarted","Data":"d852575caec07fb2deb658688a817553e7630d7290b522a04cc78f45bc54b11e"}
Oct 30 18:45:41.657185 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:41.657162    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"e8fb799e-99c0-41b4-885c-00946a448ea5","Type":"ContainerStarted","Data":"6114c97dcafb12bcb7d38926c31608ce793cc6c35400aa4fd1f80c31316781e5"}
Oct 30 18:45:42.008840 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:42.008801    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06ab042d-6af1-47f0-8677-7473519b3e1d" path="/var/lib/kubelet/pods/06ab042d-6af1-47f0-8677-7473519b3e1d/volumes"
Oct 30 18:45:42.664248 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:42.664212    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"e8fb799e-99c0-41b4-885c-00946a448ea5","Type":"ContainerStarted","Data":"8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668"}
Oct 30 18:45:42.796003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:42.795965    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"
Oct 30 18:45:42.801193 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:45:42.801168    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:45:42.833868 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:42.833814    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-1" podStartSLOduration=2.833802931 podStartE2EDuration="2.833802931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:42.718846562 +0000 UTC m=+2912.885664689" watchObservedRunningTime="2026-10-30 18:45:42.833802931 +0000 UTC m=+2913.000621058"
Oct 30 18:45:43.259852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.259820    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"]
Oct 30 18:45:43.263598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.263572    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"]
Oct 30 18:45:43.329638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.329610    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"]
Oct 30 18:45:43.329955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.329934    5746 cpu_manager.go:414] "containerMap: removing container" podUID="b6c00844-e44d-4541-98db-538abac52cb2" containerName="curl"
Oct 30 18:45:43.329955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.329949    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6c00844-e44d-4541-98db-538abac52cb2" containerName="curl"
Oct 30 18:45:43.330051 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.330012    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6c00844-e44d-4541-98db-538abac52cb2" containerName="curl"
Oct 30 18:45:43.332681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.332663    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"
Oct 30 18:45:43.343031 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.343012    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"
Oct 30 18:45:43.350373 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.350345    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"]
Oct 30 18:45:43.638558 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.638523    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"]
Oct 30 18:45:43.642040 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:45:43.642013    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb42bba15_da05_41b7_a0de_2376da1c78d7.slice/crio-dca03757fa878fd6781c4bd6a5fa9aea6d844786ab0ab1d0e6cf866ba47fbd31 WatchSource:0}: Error finding container dca03757fa878fd6781c4bd6a5fa9aea6d844786ab0ab1d0e6cf866ba47fbd31: Status 404 returned error can't find the container with id dca03757fa878fd6781c4bd6a5fa9aea6d844786ab0ab1d0e6cf866ba47fbd31
Oct 30 18:45:43.671165 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.671138    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397" event={"ID":"b42bba15-da05-41b7-a0de-2376da1c78d7","Type":"ContainerStarted","Data":"dca03757fa878fd6781c4bd6a5fa9aea6d844786ab0ab1d0e6cf866ba47fbd31"}
Oct 30 18:45:43.672376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.672355    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"
Oct 30 18:45:43.672492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:43.672377    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d852575caec07fb2deb658688a817553e7630d7290b522a04cc78f45bc54b11e"
Oct 30 18:45:43.697938 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:45:43.697892    5746 status_manager.go:1045] "Failed to get status for pod" err="pods \"thanos-ingress-prometheus-18e36303008866a5\" is forbidden: User \"system:node:ip-10-0-131-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"redhat-ods-monitoring\": no relationship found between node 'ip-10-0-131-31.ec2.internal' and this object" podUID="b6c00844-e44d-4541-98db-538abac52cb2" pod="redhat-ods-monitoring/thanos-ingress-prometheus-18e36303008866a5"
Oct 30 18:45:44.008202 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:44.008113    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6c00844-e44d-4541-98db-538abac52cb2" path="/var/lib/kubelet/pods/b6c00844-e44d-4541-98db-538abac52cb2/volumes"
Oct 30 18:45:44.677734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:44.677702    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397" event={"ID":"b42bba15-da05-41b7-a0de-2376da1c78d7","Type":"ContainerStarted","Data":"ef3c8817e40f853b2878d8872a3c26570ea3a9ad51a7fda6fbf50d82cdd809ce"}
Oct 30 18:45:49.698359 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:49.698325    5746 generic.go:356] "Generic (PLEG): container finished" podID="b42bba15-da05-41b7-a0de-2376da1c78d7" containerID="ef3c8817e40f853b2878d8872a3c26570ea3a9ad51a7fda6fbf50d82cdd809ce" exitCode=0
Oct 30 18:45:49.698781 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:49.698397    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397" event={"ID":"b42bba15-da05-41b7-a0de-2376da1c78d7","Type":"ContainerDied","Data":"ef3c8817e40f853b2878d8872a3c26570ea3a9ad51a7fda6fbf50d82cdd809ce"}
Oct 30 18:45:50.824216 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:50.824193    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"
Oct 30 18:45:51.416560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.416528    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"]
Oct 30 18:45:51.422650 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.422622    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"]
Oct 30 18:45:51.487147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.487114    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"]
Oct 30 18:45:51.487450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.487437    5746 cpu_manager.go:414] "containerMap: removing container" podUID="b42bba15-da05-41b7-a0de-2376da1c78d7" containerName="curl"
Oct 30 18:45:51.487488 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.487451    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b42bba15-da05-41b7-a0de-2376da1c78d7" containerName="curl"
Oct 30 18:45:51.487522 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.487510    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="b42bba15-da05-41b7-a0de-2376da1c78d7" containerName="curl"
Oct 30 18:45:51.490123 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.490105    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"
Oct 30 18:45:51.500356 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.500337    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"
Oct 30 18:45:51.507577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.507555    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"]
Oct 30 18:45:51.713474 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.713397    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"
Oct 30 18:45:51.713630 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.713404    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dca03757fa878fd6781c4bd6a5fa9aea6d844786ab0ab1d0e6cf866ba47fbd31"
Oct 30 18:45:51.734471 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:45:51.734440    5746 status_manager.go:1045] "Failed to get status for pod" err="pods \"thanos-ingress-perses-18e363037e561397\" is forbidden: User \"system:node:ip-10-0-131-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"redhat-ods-monitoring\": no relationship found between node 'ip-10-0-131-31.ec2.internal' and this object" podUID="b42bba15-da05-41b7-a0de-2376da1c78d7" pod="redhat-ods-monitoring/thanos-ingress-perses-18e363037e561397"
Oct 30 18:45:51.793764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:51.793726    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"]
Oct 30 18:45:51.797538 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:45:51.797507    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e35866f_96b6_4841_9f4b_d5e3a1b7d758.slice/crio-37bf2397fb193d90c4892fcc161a5651c2dd0dae59188ffbdb527653ae78adf9 WatchSource:0}: Error finding container 37bf2397fb193d90c4892fcc161a5651c2dd0dae59188ffbdb527653ae78adf9: Status 404 returned error can't find the container with id 37bf2397fb193d90c4892fcc161a5651c2dd0dae59188ffbdb527653ae78adf9
Oct 30 18:45:52.008647 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:52.008565    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b42bba15-da05-41b7-a0de-2376da1c78d7" path="/var/lib/kubelet/pods/b42bba15-da05-41b7-a0de-2376da1c78d7/volumes"
Oct 30 18:45:52.718791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:52.718755    5746 generic.go:356] "Generic (PLEG): container finished" podID="8e35866f-96b6-4841-9f4b-d5e3a1b7d758" containerID="afa726d6eabf8ca25021f59f73fdbd2184897957442ddfa9b567603d3d94d375" exitCode=0
Oct 30 18:45:52.719008 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:52.718835    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae" event={"ID":"8e35866f-96b6-4841-9f4b-d5e3a1b7d758","Type":"ContainerDied","Data":"afa726d6eabf8ca25021f59f73fdbd2184897957442ddfa9b567603d3d94d375"}
Oct 30 18:45:52.719008 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:52.718870    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae" event={"ID":"8e35866f-96b6-4841-9f4b-d5e3a1b7d758","Type":"ContainerStarted","Data":"37bf2397fb193d90c4892fcc161a5651c2dd0dae59188ffbdb527653ae78adf9"}
Oct 30 18:45:53.845598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:53.845573    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"
Oct 30 18:45:54.728327 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:54.728295    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"
Oct 30 18:45:54.728327 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:54.728316    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae" event={"ID":"8e35866f-96b6-4841-9f4b-d5e3a1b7d758","Type":"ContainerDied","Data":"37bf2397fb193d90c4892fcc161a5651c2dd0dae59188ffbdb527653ae78adf9"}
Oct 30 18:45:54.728534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:54.728349    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37bf2397fb193d90c4892fcc161a5651c2dd0dae59188ffbdb527653ae78adf9"
Oct 30 18:45:55.544295 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.544268    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"]
Oct 30 18:45:55.548057 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.548025    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-korrel8r-18e3630564adaaae"]
Oct 30 18:45:55.631277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.631243    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"]
Oct 30 18:45:55.631525 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.631513    5746 cpu_manager.go:414] "containerMap: removing container" podUID="8e35866f-96b6-4841-9f4b-d5e3a1b7d758" containerName="curl"
Oct 30 18:45:55.631579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.631527    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e35866f-96b6-4841-9f4b-d5e3a1b7d758" containerName="curl"
Oct 30 18:45:55.631612 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.631597    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e35866f-96b6-4841-9f4b-d5e3a1b7d758" containerName="curl"
Oct 30 18:45:55.634519 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.634501    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"
Oct 30 18:45:55.645079 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.645061    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"
Oct 30 18:45:55.650175 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.650154    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"]
Oct 30 18:45:55.939530 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:55.939500    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"]
Oct 30 18:45:55.942895 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:45:55.942857    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7f1f182_a2ef_4604_82d2_a25d2f74a8cf.slice/crio-f46d3118d8eb2bdd9072ffcaf59be20367404f6ce6a3051c4281bdc6cc9b6fef WatchSource:0}: Error finding container f46d3118d8eb2bdd9072ffcaf59be20367404f6ce6a3051c4281bdc6cc9b6fef: Status 404 returned error can't find the container with id f46d3118d8eb2bdd9072ffcaf59be20367404f6ce6a3051c4281bdc6cc9b6fef
Oct 30 18:45:56.007670 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:56.007648    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e35866f-96b6-4841-9f4b-d5e3a1b7d758" path="/var/lib/kubelet/pods/8e35866f-96b6-4841-9f4b-d5e3a1b7d758/volumes"
Oct 30 18:45:56.738277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:56.738241    5746 generic.go:356] "Generic (PLEG): container finished" podID="e7f1f182-a2ef-4604-82d2-a25d2f74a8cf" containerID="d04e4e3fa427b48ef038d0daeea4178b8f30a3ad02912f14d7adca1db53553db" exitCode=0
Oct 30 18:45:56.738710 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:56.738323    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3" event={"ID":"e7f1f182-a2ef-4604-82d2-a25d2f74a8cf","Type":"ContainerDied","Data":"d04e4e3fa427b48ef038d0daeea4178b8f30a3ad02912f14d7adca1db53553db"}
Oct 30 18:45:56.738710 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:56.738372    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3" event={"ID":"e7f1f182-a2ef-4604-82d2-a25d2f74a8cf","Type":"ContainerStarted","Data":"f46d3118d8eb2bdd9072ffcaf59be20367404f6ce6a3051c4281bdc6cc9b6fef"}
Oct 30 18:45:57.868113 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:57.868087    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"
Oct 30 18:45:58.752286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:58.752260    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"
Oct 30 18:45:58.752286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:58.752268    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3" event={"ID":"e7f1f182-a2ef-4604-82d2-a25d2f74a8cf","Type":"ContainerDied","Data":"f46d3118d8eb2bdd9072ffcaf59be20367404f6ce6a3051c4281bdc6cc9b6fef"}
Oct 30 18:45:58.752497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:58.752305    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f46d3118d8eb2bdd9072ffcaf59be20367404f6ce6a3051c4281bdc6cc9b6fef"
Oct 30 18:45:59.690239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:59.690204    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"]
Oct 30 18:45:59.694564 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:45:59.694537    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-ingress-authenticated-proxy-18e363065bb973f3"]
Oct 30 18:46:00.008636 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.008556    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7f1f182-a2ef-4604-82d2-a25d2f74a8cf" path="/var/lib/kubelet/pods/e7f1f182-a2ef-4604-82d2-a25d2f74a8cf/volumes"
Oct 30 18:46:00.766420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.766391    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"]
Oct 30 18:46:00.766833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.766670    5746 cpu_manager.go:414] "containerMap: removing container" podUID="e7f1f182-a2ef-4604-82d2-a25d2f74a8cf" containerName="curl"
Oct 30 18:46:00.766833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.766681    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7f1f182-a2ef-4604-82d2-a25d2f74a8cf" containerName="curl"
Oct 30 18:46:00.766833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.766736    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7f1f182-a2ef-4604-82d2-a25d2f74a8cf" containerName="curl"
Oct 30 18:46:00.769719 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.769701    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"
Oct 30 18:46:00.784306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.784282    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"thanos-ingress-probe-18e3630750d881d3\"/\"default-dockercfg-q26lq\""
Oct 30 18:46:00.785114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.785094    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"]
Oct 30 18:46:00.791181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:00.791162    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"
Oct 30 18:46:01.087935 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:01.087515    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"]
Oct 30 18:46:01.092493 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:01.092462    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod896758d9_8c14_4815_b98f_901dadd25085.slice/crio-f6e493d3bdfbea008a7785495048ad223c06e18ed8d2a50cb32e881a009715b1 WatchSource:0}: Error finding container f6e493d3bdfbea008a7785495048ad223c06e18ed8d2a50cb32e881a009715b1: Status 404 returned error can't find the container with id f6e493d3bdfbea008a7785495048ad223c06e18ed8d2a50cb32e881a009715b1
Oct 30 18:46:01.766973 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:01.766936    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0" event={"ID":"896758d9-8c14-4815-b98f-901dadd25085","Type":"ContainerStarted","Data":"23380b91c8afe7cf2c25884033a846d9796555b3d57dc9f4343a6aa440f92060"}
Oct 30 18:46:01.767406 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:01.766979    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0" event={"ID":"896758d9-8c14-4815-b98f-901dadd25085","Type":"ContainerStarted","Data":"f6e493d3bdfbea008a7785495048ad223c06e18ed8d2a50cb32e881a009715b1"}
Oct 30 18:46:01.804767 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:01.804712    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0" podStartSLOduration=1.804700594 podStartE2EDuration="1.804700594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:01.804317994 +0000 UTC m=+2931.971136120" watchObservedRunningTime="2026-10-30 18:46:01.804700594 +0000 UTC m=+2931.971518721"
Oct 30 18:46:06.788590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:06.788553    5746 generic.go:356] "Generic (PLEG): container finished" podID="896758d9-8c14-4815-b98f-901dadd25085" containerID="23380b91c8afe7cf2c25884033a846d9796555b3d57dc9f4343a6aa440f92060" exitCode=0
Oct 30 18:46:06.789107 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:06.788615    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0" event={"ID":"896758d9-8c14-4815-b98f-901dadd25085","Type":"ContainerDied","Data":"23380b91c8afe7cf2c25884033a846d9796555b3d57dc9f4343a6aa440f92060"}
Oct 30 18:46:07.165434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.165398    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"]
Oct 30 18:46:07.168574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.168552    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.195432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.195405    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"]
Oct 30 18:46:07.228028 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.227997    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.228227 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.228041    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rc9v\" (UniqueName: \"kubernetes.io/projected/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-kube-api-access-7rc9v\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.228227 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.228137    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-ta-internal\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.329081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.329035    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.329286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.329097    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rc9v\" (UniqueName: \"kubernetes.io/projected/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-kube-api-access-7rc9v\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.329286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.329149    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-ta-internal\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.329836 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.329809    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-ta-internal\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.331672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.331652    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.352465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.352430    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rc9v\" (UniqueName: \"kubernetes.io/projected/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-kube-api-access-7rc9v\") pod \"data-science-collector-targetallocator-fb898c88-gpplz\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.480054 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.479961    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:07.774926 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.774825    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"]
Oct 30 18:46:07.778632 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:07.778601    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded3e2f6e_d6e2_4884_a876_aacf41a5bebf.slice/crio-3bd477ac3d039bc1ece6704cafaf8a7b4f433e9b53fd26b24223bfd4a6617b05 WatchSource:0}: Error finding container 3bd477ac3d039bc1ece6704cafaf8a7b4f433e9b53fd26b24223bfd4a6617b05: Status 404 returned error can't find the container with id 3bd477ac3d039bc1ece6704cafaf8a7b4f433e9b53fd26b24223bfd4a6617b05
Oct 30 18:46:07.780794 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.780759    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:07.780895 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.780831    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:07.793372 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.793349    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz" event={"ID":"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf","Type":"ContainerStarted","Data":"3bd477ac3d039bc1ece6704cafaf8a7b4f433e9b53fd26b24223bfd4a6617b05"}
Oct 30 18:46:07.914656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:07.914631    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"
Oct 30 18:46:08.799863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.799822    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz" event={"ID":"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf","Type":"ContainerStarted","Data":"f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316"}
Oct 30 18:46:08.800377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.799877    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:08.801219 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.801202    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"
Oct 30 18:46:08.801320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.801212    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0" event={"ID":"896758d9-8c14-4815-b98f-901dadd25085","Type":"ContainerDied","Data":"f6e493d3bdfbea008a7785495048ad223c06e18ed8d2a50cb32e881a009715b1"}
Oct 30 18:46:08.801320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.801245    5746 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6e493d3bdfbea008a7785495048ad223c06e18ed8d2a50cb32e881a009715b1"
Oct 30 18:46:08.802347 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.802327    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:08.842081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.842034    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz" podStartSLOduration=1.842020429 podStartE2EDuration="1.842020429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:08.840818706 +0000 UTC m=+2939.007636844" watchObservedRunningTime="2026-10-30 18:46:08.842020429 +0000 UTC m=+2939.008838552"
Oct 30 18:46:08.857959 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.857933    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"]
Oct 30 18:46:08.860054 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.860031    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["thanos-ingress-probe-18e3630750d881d3/thanos-ingress-untrusted-18e3630751debca0"]
Oct 30 18:46:08.928928 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.928885    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"]
Oct 30 18:46:08.929135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:08.929111    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw" podUID="ad372e46-c110-44ab-9726-d8b24c10d4c6" containerName="ta-container" containerID="cri-o://67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd" gracePeriod=30
Oct 30 18:46:09.172456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.172434    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:46:09.246532 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.246500    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad372e46-c110-44ab-9726-d8b24c10d4c6-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ad372e46-c110-44ab-9726-d8b24c10d4c6-data-science-collector-ta-server-cert\") pod \"ad372e46-c110-44ab-9726-d8b24c10d4c6\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") "
Oct 30 18:46:09.246718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.246560    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad372e46-c110-44ab-9726-d8b24c10d4c6-ta-internal\" (UniqueName: \"kubernetes.io/configmap/ad372e46-c110-44ab-9726-d8b24c10d4c6-ta-internal\") pod \"ad372e46-c110-44ab-9726-d8b24c10d4c6\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") "
Oct 30 18:46:09.246718 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.246600    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad372e46-c110-44ab-9726-d8b24c10d4c6-kube-api-access-sqq98\" (UniqueName: \"kubernetes.io/projected/ad372e46-c110-44ab-9726-d8b24c10d4c6-kube-api-access-sqq98\") pod \"ad372e46-c110-44ab-9726-d8b24c10d4c6\" (UID: \"ad372e46-c110-44ab-9726-d8b24c10d4c6\") "
Oct 30 18:46:09.247019 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.246993    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad372e46-c110-44ab-9726-d8b24c10d4c6-ta-internal" pod "ad372e46-c110-44ab-9726-d8b24c10d4c6" (UID: "ad372e46-c110-44ab-9726-d8b24c10d4c6"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:09.248660 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.248633    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad372e46-c110-44ab-9726-d8b24c10d4c6-data-science-collector-ta-server-cert" pod "ad372e46-c110-44ab-9726-d8b24c10d4c6" (UID: "ad372e46-c110-44ab-9726-d8b24c10d4c6"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:09.249190 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.249168    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad372e46-c110-44ab-9726-d8b24c10d4c6-kube-api-access-sqq98" pod "ad372e46-c110-44ab-9726-d8b24c10d4c6" (UID: "ad372e46-c110-44ab-9726-d8b24c10d4c6"). InnerVolumeSpecName "kube-api-access-sqq98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:09.347350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.347325    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ad372e46-c110-44ab-9726-d8b24c10d4c6-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:09.347350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.347349    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqq98\" (UniqueName: \"kubernetes.io/projected/ad372e46-c110-44ab-9726-d8b24c10d4c6-kube-api-access-sqq98\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:09.347541 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.347359    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ad372e46-c110-44ab-9726-d8b24c10d4c6-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:09.807172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.807075    5746 generic.go:356] "Generic (PLEG): container finished" podID="ad372e46-c110-44ab-9726-d8b24c10d4c6" containerID="67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd" exitCode=0
Oct 30 18:46:09.807172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.807123    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"
Oct 30 18:46:09.807680 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.807162    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw" event={"ID":"ad372e46-c110-44ab-9726-d8b24c10d4c6","Type":"ContainerDied","Data":"67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd"}
Oct 30 18:46:09.807680 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.807209    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw" event={"ID":"ad372e46-c110-44ab-9726-d8b24c10d4c6","Type":"ContainerDied","Data":"669af8f8f5be38a947146b903d1578ea9d602cf0287d0e1fc91537f5056e51ea"}
Oct 30 18:46:09.807680 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.807232    5746 scope.go:122] "RemoveContainer" containerID="67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd"
Oct 30 18:46:09.820459 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.820244    5746 scope.go:122] "RemoveContainer" containerID="67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd"
Oct 30 18:46:09.820618 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:09.820587    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd\": container with ID starting with 67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd not found: ID does not exist" containerID="67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd"
Oct 30 18:46:09.820728 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.820629    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd"} err="failed to get container status \"67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd\": rpc error: code = NotFound desc = could not find container \"67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd\": container with ID starting with 67428b7cd6eea7e447ec4a12e79de74edf0d87d9f1f4b67342bafe73e4dcc8cd not found: ID does not exist"
Oct 30 18:46:09.856919 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.856875    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"]
Oct 30 18:46:09.868628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:09.868606    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-kdxjw"]
Oct 30 18:46:10.008326 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:10.008294    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="896758d9-8c14-4815-b98f-901dadd25085" path="/var/lib/kubelet/pods/896758d9-8c14-4815-b98f-901dadd25085/volumes"
Oct 30 18:46:10.008647 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:10.008632    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad372e46-c110-44ab-9726-d8b24c10d4c6" path="/var/lib/kubelet/pods/ad372e46-c110-44ab-9726-d8b24c10d4c6/volumes"
Oct 30 18:46:12.790459 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:12.790424    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:46:14.376789 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.376748    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"]
Oct 30 18:46:14.377437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.377190    5746 cpu_manager.go:414] "containerMap: removing container" podUID="896758d9-8c14-4815-b98f-901dadd25085" containerName="curl"
Oct 30 18:46:14.377437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.377209    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="896758d9-8c14-4815-b98f-901dadd25085" containerName="curl"
Oct 30 18:46:14.377437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.377224    5746 cpu_manager.go:414] "containerMap: removing container" podUID="ad372e46-c110-44ab-9726-d8b24c10d4c6" containerName="ta-container"
Oct 30 18:46:14.377437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.377232    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad372e46-c110-44ab-9726-d8b24c10d4c6" containerName="ta-container"
Oct 30 18:46:14.377437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.377306    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="896758d9-8c14-4815-b98f-901dadd25085" containerName="curl"
Oct 30 18:46:14.377437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.377321    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad372e46-c110-44ab-9726-d8b24c10d4c6" containerName="ta-container"
Oct 30 18:46:14.381706 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.381687    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.400523 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.400498    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"]
Oct 30 18:46:14.491133 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.491095    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt65n\" (UniqueName: \"kubernetes.io/projected/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-kube-api-access-xt65n\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.491133 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.491134    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.491358 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.491168    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-ta-internal\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.591824 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.591792    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt65n\" (UniqueName: \"kubernetes.io/projected/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-kube-api-access-xt65n\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.591824 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.591825    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.592047 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.591858    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-ta-internal\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.592581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.592564    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-ta-internal\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.594602 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.594585    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.615037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.615005    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt65n\" (UniqueName: \"kubernetes.io/projected/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-kube-api-access-xt65n\") pod \"data-science-collector-targetallocator-658494f955-jc2vw\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.692145 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.692057    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:14.990007 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.989930    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"]
Oct 30 18:46:14.993452 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:14.993419    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9f7465b_bbe6_4c38_8594_9acc1e17fc0f.slice/crio-0945be814aedf1dcb9ca660f5001badbdd40c204ce42ca0a291486cdce9618b0 WatchSource:0}: Error finding container 0945be814aedf1dcb9ca660f5001badbdd40c204ce42ca0a291486cdce9618b0: Status 404 returned error can't find the container with id 0945be814aedf1dcb9ca660f5001badbdd40c204ce42ca0a291486cdce9618b0
Oct 30 18:46:14.995615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.995582    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:14.995692 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:14.995638    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:15.832987 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:15.832948    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw" event={"ID":"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f","Type":"ContainerStarted","Data":"8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004"}
Oct 30 18:46:15.832987 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:15.832989    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw" event={"ID":"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f","Type":"ContainerStarted","Data":"0945be814aedf1dcb9ca660f5001badbdd40c204ce42ca0a291486cdce9618b0"}
Oct 30 18:46:15.833525 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:15.833175    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:15.834853 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:15.834831    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:15.875788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:15.875727    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw" podStartSLOduration=1.875712584 podStartE2EDuration="1.875712584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:15.874560216 +0000 UTC m=+2946.041378331" watchObservedRunningTime="2026-10-30 18:46:15.875712584 +0000 UTC m=+2946.042530712"
Oct 30 18:46:15.945768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:15.945737    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"]
Oct 30 18:46:15.945962 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:15.945917    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz" podUID="ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" containerName="ta-container" containerID="cri-o://f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316" gracePeriod=30
Oct 30 18:46:16.185376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.185353    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:16.306932 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.306855    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-kube-api-access-7rc9v\" (UniqueName: \"kubernetes.io/projected/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-kube-api-access-7rc9v\") pod \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") "
Oct 30 18:46:16.307108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.306986    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-data-science-collector-ta-server-cert\") pod \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") "
Oct 30 18:46:16.307108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.307043    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-ta-internal\" (UniqueName: \"kubernetes.io/configmap/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-ta-internal\") pod \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\" (UID: \"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf\") "
Oct 30 18:46:16.307456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.307415    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-ta-internal" pod "ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" (UID: "ed3e2f6e-d6e2-4884-a876-aacf41a5bebf"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:16.309124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.309103    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-data-science-collector-ta-server-cert" pod "ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" (UID: "ed3e2f6e-d6e2-4884-a876-aacf41a5bebf"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:16.309197 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.309122    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-kube-api-access-7rc9v" pod "ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" (UID: "ed3e2f6e-d6e2-4884-a876-aacf41a5bebf"). InnerVolumeSpecName "kube-api-access-7rc9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:16.408505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.408422    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:16.408505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.408451    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:16.408505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.408463    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rc9v\" (UniqueName: \"kubernetes.io/projected/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf-kube-api-access-7rc9v\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:16.838401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.838311    5746 generic.go:356] "Generic (PLEG): container finished" podID="ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" containerID="f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316" exitCode=0
Oct 30 18:46:16.838401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.838356    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"
Oct 30 18:46:16.838994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.838398    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz" event={"ID":"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf","Type":"ContainerDied","Data":"f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316"}
Oct 30 18:46:16.838994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.838435    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz" event={"ID":"ed3e2f6e-d6e2-4884-a876-aacf41a5bebf","Type":"ContainerDied","Data":"3bd477ac3d039bc1ece6704cafaf8a7b4f433e9b53fd26b24223bfd4a6617b05"}
Oct 30 18:46:16.838994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.838455    5746 scope.go:122] "RemoveContainer" containerID="f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316"
Oct 30 18:46:16.847272 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.847255    5746 scope.go:122] "RemoveContainer" containerID="f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316"
Oct 30 18:46:16.847555 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:16.847539    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316\": container with ID starting with f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316 not found: ID does not exist" containerID="f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316"
Oct 30 18:46:16.847618 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.847567    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316"} err="failed to get container status \"f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316\": rpc error: code = NotFound desc = could not find container \"f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316\": container with ID starting with f6b29e88f8985f18a38485f9138289ea8a0f32eec10e7e97860e23cf2fe76316 not found: ID does not exist"
Oct 30 18:46:16.888090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.888065    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"]
Oct 30 18:46:16.899785 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:16.899746    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-fb898c88-gpplz"]
Oct 30 18:46:18.008257 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:18.008227    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" path="/var/lib/kubelet/pods/ed3e2f6e-d6e2-4884-a876-aacf41a5bebf/volumes"
Oct 30 18:46:23.960363 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:23.960327    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:23.961423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:23.960555    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-1" podUID="e8fb799e-99c0-41b4-885c-00946a448ea5" containerName="otc-container" containerID="cri-o://8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668" gracePeriod=30
Oct 30 18:46:24.199723 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.199686    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"]
Oct 30 18:46:24.200012 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.199998    5746 cpu_manager.go:414] "containerMap: removing container" podUID="ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" containerName="ta-container"
Oct 30 18:46:24.200067 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.200013    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" containerName="ta-container"
Oct 30 18:46:24.200118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.200089    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed3e2f6e-d6e2-4884-a876-aacf41a5bebf" containerName="ta-container"
Oct 30 18:46:24.204275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.204254    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.224781 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.224723    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"]
Oct 30 18:46:24.272811 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.272776    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/399ba7f1-b339-460f-bd23-10413c751dc8-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.273021 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.272827    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7lmb\" (UniqueName: \"kubernetes.io/projected/399ba7f1-b339-460f-bd23-10413c751dc8-kube-api-access-m7lmb\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.273021 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.272976    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/399ba7f1-b339-460f-bd23-10413c751dc8-ta-internal\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.373970 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.373930    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/399ba7f1-b339-460f-bd23-10413c751dc8-ta-internal\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.374136 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.373987    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/399ba7f1-b339-460f-bd23-10413c751dc8-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.374136 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.374015    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7lmb\" (UniqueName: \"kubernetes.io/projected/399ba7f1-b339-460f-bd23-10413c751dc8-kube-api-access-m7lmb\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.374829 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.374801    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/399ba7f1-b339-460f-bd23-10413c751dc8-ta-internal\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.376716 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.376696    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/399ba7f1-b339-460f-bd23-10413c751dc8-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.397105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.397079    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7lmb\" (UniqueName: \"kubernetes.io/projected/399ba7f1-b339-460f-bd23-10413c751dc8-kube-api-access-m7lmb\") pod \"data-science-collector-targetallocator-5f6bb744cd-7fjtp\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.515614 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.515537    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.810030 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.809945    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"]
Oct 30 18:46:24.813842 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:24.813813    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod399ba7f1_b339_460f_bd23_10413c751dc8.slice/crio-d4dc6aa7aa4fd0cd1dea65c2a87a47506d5e01176b0748ab373b9340b9284872 WatchSource:0}: Error finding container d4dc6aa7aa4fd0cd1dea65c2a87a47506d5e01176b0748ab373b9340b9284872: Status 404 returned error can't find the container with id d4dc6aa7aa4fd0cd1dea65c2a87a47506d5e01176b0748ab373b9340b9284872
Oct 30 18:46:24.816168 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.816134    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:24.816297 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.816209    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:24.874689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.874657    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp" event={"ID":"399ba7f1-b339-460f-bd23-10413c751dc8","Type":"ContainerStarted","Data":"71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338"}
Oct 30 18:46:24.874789 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.874702    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp" event={"ID":"399ba7f1-b339-460f-bd23-10413c751dc8","Type":"ContainerStarted","Data":"d4dc6aa7aa4fd0cd1dea65c2a87a47506d5e01176b0748ab373b9340b9284872"}
Oct 30 18:46:24.874854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.874836    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:24.876389 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.876362    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp" podUID="399ba7f1-b339-460f-bd23-10413c751dc8" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.125:8080/readyz\": dial tcp 10.133.0.125:8080: connect: connection refused"
Oct 30 18:46:24.915029 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:24.914980    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp" podStartSLOduration=0.914938807 podStartE2EDuration="914.938807ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:24.914607567 +0000 UTC m=+2955.081425716" watchObservedRunningTime="2026-10-30 18:46:24.914938807 +0000 UTC m=+2955.081756928"
Oct 30 18:46:25.879623 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:25.879596    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:25.959673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:25.959644    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"]
Oct 30 18:46:25.959851 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:25.959813    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw" podUID="f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" containerName="ta-container" containerID="cri-o://8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004" gracePeriod=30
Oct 30 18:46:26.203499 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.203474    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:26.289432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.289400    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-ta-internal\" (UniqueName: \"kubernetes.io/configmap/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-ta-internal\") pod \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") "
Oct 30 18:46:26.289607 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.289438    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-data-science-collector-ta-server-cert\") pod \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") "
Oct 30 18:46:26.289607 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.289493    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-kube-api-access-xt65n\" (UniqueName: \"kubernetes.io/projected/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-kube-api-access-xt65n\") pod \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\" (UID: \"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f\") "
Oct 30 18:46:26.289793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.289763    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-ta-internal" pod "f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" (UID: "f9f7465b-bbe6-4c38-8594-9acc1e17fc0f"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:26.291713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.291681    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-kube-api-access-xt65n" pod "f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" (UID: "f9f7465b-bbe6-4c38-8594-9acc1e17fc0f"). InnerVolumeSpecName "kube-api-access-xt65n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:26.291713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.291692    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-data-science-collector-ta-server-cert" pod "f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" (UID: "f9f7465b-bbe6-4c38-8594-9acc1e17fc0f"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:26.303774 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.303754    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:26.390617 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390557    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-tls-certs\") pod \"e8fb799e-99c0-41b4-885c-00946a448ea5\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") "
Oct 30 18:46:26.390617 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390596    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-otlp-tls-certs\") pod \"e8fb799e-99c0-41b4-885c-00946a448ea5\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") "
Oct 30 18:46:26.390785 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390635    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8fb799e-99c0-41b4-885c-00946a448ea5-kube-api-access-qbmpz\" (UniqueName: \"kubernetes.io/projected/e8fb799e-99c0-41b4-885c-00946a448ea5-kube-api-access-qbmpz\") pod \"e8fb799e-99c0-41b4-885c-00946a448ea5\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") "
Oct 30 18:46:26.390785 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390665    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8fb799e-99c0-41b4-885c-00946a448ea5-otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8fb799e-99c0-41b4-885c-00946a448ea5-otc-internal\") pod \"e8fb799e-99c0-41b4-885c-00946a448ea5\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") "
Oct 30 18:46:26.390785 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390704    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-monitoring-tls-certs\") pod \"e8fb799e-99c0-41b4-885c-00946a448ea5\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") "
Oct 30 18:46:26.390785 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390726    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-data-science-collector-ta-client-cert\") pod \"e8fb799e-99c0-41b4-885c-00946a448ea5\" (UID: \"e8fb799e-99c0-41b4-885c-00946a448ea5\") "
Oct 30 18:46:26.391023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390942    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.391023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390963    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.391023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.390978    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt65n\" (UniqueName: \"kubernetes.io/projected/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f-kube-api-access-xt65n\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.391207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.391177    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8fb799e-99c0-41b4-885c-00946a448ea5-otc-internal" pod "e8fb799e-99c0-41b4-885c-00946a448ea5" (UID: "e8fb799e-99c0-41b4-885c-00946a448ea5"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:26.392968 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.392941    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-tls-certs" pod "e8fb799e-99c0-41b4-885c-00946a448ea5" (UID: "e8fb799e-99c0-41b4-885c-00946a448ea5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:26.393400 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.393373    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-otlp-tls-certs" pod "e8fb799e-99c0-41b4-885c-00946a448ea5" (UID: "e8fb799e-99c0-41b4-885c-00946a448ea5"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:26.393496 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.393395    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fb799e-99c0-41b4-885c-00946a448ea5-kube-api-access-qbmpz" pod "e8fb799e-99c0-41b4-885c-00946a448ea5" (UID: "e8fb799e-99c0-41b4-885c-00946a448ea5"). InnerVolumeSpecName "kube-api-access-qbmpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:26.393712 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.393688    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-data-science-collector-ta-client-cert" pod "e8fb799e-99c0-41b4-885c-00946a448ea5" (UID: "e8fb799e-99c0-41b4-885c-00946a448ea5"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:26.393790 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.393678    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-monitoring-tls-certs" pod "e8fb799e-99c0-41b4-885c-00946a448ea5" (UID: "e8fb799e-99c0-41b4-885c-00946a448ea5"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:26.492147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.492114    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.492147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.492143    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.492332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.492154    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbmpz\" (UniqueName: \"kubernetes.io/projected/e8fb799e-99c0-41b4-885c-00946a448ea5-kube-api-access-qbmpz\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.492332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.492164    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e8fb799e-99c0-41b4-885c-00946a448ea5-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.492332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.492172    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.492332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.492181    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/e8fb799e-99c0-41b4-885c-00946a448ea5-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:26.883642 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.883608    5746 generic.go:356] "Generic (PLEG): container finished" podID="f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" containerID="8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004" exitCode=0
Oct 30 18:46:26.884181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.883648    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"
Oct 30 18:46:26.884181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.883689    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw" event={"ID":"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f","Type":"ContainerDied","Data":"8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004"}
Oct 30 18:46:26.884181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.883754    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw" event={"ID":"f9f7465b-bbe6-4c38-8594-9acc1e17fc0f","Type":"ContainerDied","Data":"0945be814aedf1dcb9ca660f5001badbdd40c204ce42ca0a291486cdce9618b0"}
Oct 30 18:46:26.884181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.883778    5746 scope.go:122] "RemoveContainer" containerID="8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004"
Oct 30 18:46:26.885197 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.885179    5746 generic.go:356] "Generic (PLEG): container finished" podID="e8fb799e-99c0-41b4-885c-00946a448ea5" containerID="8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668" exitCode=0
Oct 30 18:46:26.885296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.885228    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:26.885296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.885272    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"e8fb799e-99c0-41b4-885c-00946a448ea5","Type":"ContainerDied","Data":"8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668"}
Oct 30 18:46:26.885547 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.885524    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"e8fb799e-99c0-41b4-885c-00946a448ea5","Type":"ContainerDied","Data":"6114c97dcafb12bcb7d38926c31608ce793cc6c35400aa4fd1f80c31316781e5"}
Oct 30 18:46:26.893085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.893066    5746 scope.go:122] "RemoveContainer" containerID="8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004"
Oct 30 18:46:26.893379 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:26.893362    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004\": container with ID starting with 8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004 not found: ID does not exist" containerID="8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004"
Oct 30 18:46:26.893424 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.893388    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004"} err="failed to get container status \"8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004\": rpc error: code = NotFound desc = could not find container \"8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004\": container with ID starting with 8cbae8a511d3dcdffc65a64b319b8eaa013e9a3b147a271d629bc22443cff004 not found: ID does not exist"
Oct 30 18:46:26.893424 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.893405    5746 scope.go:122] "RemoveContainer" containerID="8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668"
Oct 30 18:46:26.901311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.901296    5746 scope.go:122] "RemoveContainer" containerID="8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668"
Oct 30 18:46:26.901568 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:26.901550    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668\": container with ID starting with 8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668 not found: ID does not exist" containerID="8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668"
Oct 30 18:46:26.901633 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.901572    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668"} err="failed to get container status \"8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668\": rpc error: code = NotFound desc = could not find container \"8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668\": container with ID starting with 8180fb5850d1ffe8a9e229cd37ad8755c146242e1ceb8a67dde2c9744bc03668 not found: ID does not exist"
Oct 30 18:46:26.933763 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.933742    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"]
Oct 30 18:46:26.945859 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.945839    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-658494f955-jc2vw"]
Oct 30 18:46:26.972535 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.972512    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:26.976958 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:26.976938    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:27.018180 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.018145    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:27.018419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.018407    5746 cpu_manager.go:414] "containerMap: removing container" podUID="e8fb799e-99c0-41b4-885c-00946a448ea5" containerName="otc-container"
Oct 30 18:46:27.018515 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.018420    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8fb799e-99c0-41b4-885c-00946a448ea5" containerName="otc-container"
Oct 30 18:46:27.018515 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.018442    5746 cpu_manager.go:414] "containerMap: removing container" podUID="f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" containerName="ta-container"
Oct 30 18:46:27.018515 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.018447    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" containerName="ta-container"
Oct 30 18:46:27.018515 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.018494    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" containerName="ta-container"
Oct 30 18:46:27.018515 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.018506    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8fb799e-99c0-41b4-885c-00946a448ea5" containerName="otc-container"
Oct 30 18:46:27.022853 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.022833    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.037736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.037719    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:46:27.037829 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.037810    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-6d9d4583\""
Oct 30 18:46:27.037881 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.037819    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:46:27.037881 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.037849    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsglb\""
Oct 30 18:46:27.037965 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.037923    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:46:27.038002 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.037968    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:46:27.042300 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.042281    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:27.097353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.097327    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d4181de0-1237-40ab-bdfc-e76837eecfec-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.097504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.097357    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.097504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.097479    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.097606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.097537    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.097606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.097574    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx2xq\" (UniqueName: \"kubernetes.io/projected/d4181de0-1237-40ab-bdfc-e76837eecfec-kube-api-access-vx2xq\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.097606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.097594    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.198750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.198670    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.198750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.198705    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.198750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.198724    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx2xq\" (UniqueName: \"kubernetes.io/projected/d4181de0-1237-40ab-bdfc-e76837eecfec-kube-api-access-vx2xq\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.199027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.198752    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.199027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.198781    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d4181de0-1237-40ab-bdfc-e76837eecfec-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.199027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.198803    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.199777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.199540    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d4181de0-1237-40ab-bdfc-e76837eecfec-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.201577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.201549    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.201661 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.201625    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.201716 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.201700    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.201774 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.201757    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.221598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.221579    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx2xq\" (UniqueName: \"kubernetes.io/projected/d4181de0-1237-40ab-bdfc-e76837eecfec-kube-api-access-vx2xq\") pod \"data-science-collector-collector-1\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.334746 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.334719    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:27.634870 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.634840    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:27.638161 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:27.638137    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4181de0_1237_40ab_bdfc_e76837eecfec.slice/crio-836c108c352f2dba7130c749e3dc8169edb0817353c5cac822c7dacdaefa9e32 WatchSource:0}: Error finding container 836c108c352f2dba7130c749e3dc8169edb0817353c5cac822c7dacdaefa9e32: Status 404 returned error can't find the container with id 836c108c352f2dba7130c749e3dc8169edb0817353c5cac822c7dacdaefa9e32
Oct 30 18:46:27.640375 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.640344    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:27.640445 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.640404    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:27.893100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.893023    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"d4181de0-1237-40ab-bdfc-e76837eecfec","Type":"ContainerStarted","Data":"c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9"}
Oct 30 18:46:27.893661 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.893097    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"d4181de0-1237-40ab-bdfc-e76837eecfec","Type":"ContainerStarted","Data":"836c108c352f2dba7130c749e3dc8169edb0817353c5cac822c7dacdaefa9e32"}
Oct 30 18:46:27.935275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:27.935228    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-1" podStartSLOduration=1.935214068 podStartE2EDuration="1.935214068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:27.934337494 +0000 UTC m=+2958.101155634" watchObservedRunningTime="2026-10-30 18:46:27.935214068 +0000 UTC m=+2958.102032194"
Oct 30 18:46:28.008544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:28.008510    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8fb799e-99c0-41b4-885c-00946a448ea5" path="/var/lib/kubelet/pods/e8fb799e-99c0-41b4-885c-00946a448ea5/volumes"
Oct 30 18:46:28.009050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:28.009029    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9f7465b-bbe6-4c38-8594-9acc1e17fc0f" path="/var/lib/kubelet/pods/f9f7465b-bbe6-4c38-8594-9acc1e17fc0f/volumes"
Oct 30 18:46:41.069848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.069815    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:41.070468 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.070042    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-1" podUID="d4181de0-1237-40ab-bdfc-e76837eecfec" containerName="otc-container" containerID="cri-o://c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9" gracePeriod=30
Oct 30 18:46:41.306958 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.306918    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"]
Oct 30 18:46:41.310336 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.310312    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.330537 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.330477    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"]
Oct 30 18:46:41.424518 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.424482    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpsn4\" (UniqueName: \"kubernetes.io/projected/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-kube-api-access-zpsn4\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.424695 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.424532    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.424695 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.424632    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-ta-internal\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.525244 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.525202    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.525244 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.525248    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-ta-internal\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.525453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.525295    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpsn4\" (UniqueName: \"kubernetes.io/projected/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-kube-api-access-zpsn4\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.526006 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.525974    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-ta-internal\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.527771 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.527749    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.548680 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.548648    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpsn4\" (UniqueName: \"kubernetes.io/projected/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-kube-api-access-zpsn4\") pod \"data-science-collector-targetallocator-f9845c957-btr6w\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.621725 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.621675    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:41.916902 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.916735    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"]
Oct 30 18:46:41.919881 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:41.919846    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b2af657_6a9e_47ec_8386_4f79c5e6fcf9.slice/crio-8859e2a4066d2b97fe2b70728a211eb39c879ee7577819fd9e8c0194b94c27d2 WatchSource:0}: Error finding container 8859e2a4066d2b97fe2b70728a211eb39c879ee7577819fd9e8c0194b94c27d2: Status 404 returned error can't find the container with id 8859e2a4066d2b97fe2b70728a211eb39c879ee7577819fd9e8c0194b94c27d2
Oct 30 18:46:41.922088 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.922058    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:41.922163 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.922117    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:41.953258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:41.953104    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w" event={"ID":"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9","Type":"ContainerStarted","Data":"8859e2a4066d2b97fe2b70728a211eb39c879ee7577819fd9e8c0194b94c27d2"}
Oct 30 18:46:42.298165 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.298142    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:42.432854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.432765    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4181de0-1237-40ab-bdfc-e76837eecfec-otc-internal\" (UniqueName: \"kubernetes.io/configmap/d4181de0-1237-40ab-bdfc-e76837eecfec-otc-internal\") pod \"d4181de0-1237-40ab-bdfc-e76837eecfec\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") "
Oct 30 18:46:42.432854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.432802    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-otlp-tls-certs\") pod \"d4181de0-1237-40ab-bdfc-e76837eecfec\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") "
Oct 30 18:46:42.432854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.432823    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-data-science-collector-ta-client-cert\") pod \"d4181de0-1237-40ab-bdfc-e76837eecfec\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") "
Oct 30 18:46:42.433154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.432883    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-tls-certs\") pod \"d4181de0-1237-40ab-bdfc-e76837eecfec\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") "
Oct 30 18:46:42.433154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.432945    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4181de0-1237-40ab-bdfc-e76837eecfec-kube-api-access-vx2xq\" (UniqueName: \"kubernetes.io/projected/d4181de0-1237-40ab-bdfc-e76837eecfec-kube-api-access-vx2xq\") pod \"d4181de0-1237-40ab-bdfc-e76837eecfec\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") "
Oct 30 18:46:42.433154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.432984    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-monitoring-tls-certs\") pod \"d4181de0-1237-40ab-bdfc-e76837eecfec\" (UID: \"d4181de0-1237-40ab-bdfc-e76837eecfec\") "
Oct 30 18:46:42.433322 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.433300    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4181de0-1237-40ab-bdfc-e76837eecfec-otc-internal" pod "d4181de0-1237-40ab-bdfc-e76837eecfec" (UID: "d4181de0-1237-40ab-bdfc-e76837eecfec"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:42.435401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.435363    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4181de0-1237-40ab-bdfc-e76837eecfec-kube-api-access-vx2xq" pod "d4181de0-1237-40ab-bdfc-e76837eecfec" (UID: "d4181de0-1237-40ab-bdfc-e76837eecfec"). InnerVolumeSpecName "kube-api-access-vx2xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:42.435517 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.435364    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-monitoring-tls-certs" pod "d4181de0-1237-40ab-bdfc-e76837eecfec" (UID: "d4181de0-1237-40ab-bdfc-e76837eecfec"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:42.435517 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.435449    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-tls-certs" pod "d4181de0-1237-40ab-bdfc-e76837eecfec" (UID: "d4181de0-1237-40ab-bdfc-e76837eecfec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:42.435742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.435717    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-data-science-collector-ta-client-cert" pod "d4181de0-1237-40ab-bdfc-e76837eecfec" (UID: "d4181de0-1237-40ab-bdfc-e76837eecfec"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:42.435846 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.435740    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-otlp-tls-certs" pod "d4181de0-1237-40ab-bdfc-e76837eecfec" (UID: "d4181de0-1237-40ab-bdfc-e76837eecfec"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:42.534248 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.534221    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:42.534338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.534257    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx2xq\" (UniqueName: \"kubernetes.io/projected/d4181de0-1237-40ab-bdfc-e76837eecfec-kube-api-access-vx2xq\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:42.534338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.534272    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:42.534338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.534285    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/d4181de0-1237-40ab-bdfc-e76837eecfec-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:42.534338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.534299    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:42.534338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.534312    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/d4181de0-1237-40ab-bdfc-e76837eecfec-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:42.790474 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:42.790377    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:46:42.958883 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.958844    5746 generic.go:356] "Generic (PLEG): container finished" podID="d4181de0-1237-40ab-bdfc-e76837eecfec" containerID="c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9" exitCode=0
Oct 30 18:46:42.959082 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.958896    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:42.959082 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.958936    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"d4181de0-1237-40ab-bdfc-e76837eecfec","Type":"ContainerDied","Data":"c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9"}
Oct 30 18:46:42.959082 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.958989    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"d4181de0-1237-40ab-bdfc-e76837eecfec","Type":"ContainerDied","Data":"836c108c352f2dba7130c749e3dc8169edb0817353c5cac822c7dacdaefa9e32"}
Oct 30 18:46:42.959082 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.959012    5746 scope.go:122] "RemoveContainer" containerID="c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9"
Oct 30 18:46:42.960638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.960614    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w" event={"ID":"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9","Type":"ContainerStarted","Data":"3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2"}
Oct 30 18:46:42.960802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.960781    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:42.962960 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.962943    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:46:42.968020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.967889    5746 scope.go:122] "RemoveContainer" containerID="c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9"
Oct 30 18:46:42.968191 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:42.968173    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9\": container with ID starting with c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9 not found: ID does not exist" containerID="c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9"
Oct 30 18:46:42.968255 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:42.968202    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9"} err="failed to get container status \"c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9\": rpc error: code = NotFound desc = could not find container \"c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9\": container with ID starting with c6aae11def067e28e918107affea67cea79ff88423382d07b4d65d9feffd1db9 not found: ID does not exist"
Oct 30 18:46:43.008400 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.008367    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:43.020141 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.020108    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:43.050827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.050752    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:43.051240 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.051221    5746 cpu_manager.go:414] "containerMap: removing container" podUID="d4181de0-1237-40ab-bdfc-e76837eecfec" containerName="otc-container"
Oct 30 18:46:43.051288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.051245    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4181de0-1237-40ab-bdfc-e76837eecfec" containerName="otc-container"
Oct 30 18:46:43.051369 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.051356    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4181de0-1237-40ab-bdfc-e76837eecfec" containerName="otc-container"
Oct 30 18:46:43.055672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.055653    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.057739 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.057700    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w" podStartSLOduration=2.057687864 podStartE2EDuration="2.057687864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:43.057178525 +0000 UTC m=+2973.223996653" watchObservedRunningTime="2026-10-30 18:46:43.057687864 +0000 UTC m=+2973.224505988"
Oct 30 18:46:43.070839 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.070807    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-a0f9182a\""
Oct 30 18:46:43.070975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.070892    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:46:43.070975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.070926    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:46:43.070975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.070956    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:46:43.070975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.070965    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsglb\""
Oct 30 18:46:43.071166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.071014    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:46:43.076771 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.076747    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:43.089179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.089154    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"]
Oct 30 18:46:43.089369 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.089330    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp" podUID="399ba7f1-b339-460f-bd23-10413c751dc8" containerName="ta-container" containerID="cri-o://71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338" gracePeriod=30
Oct 30 18:46:43.140086 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.140046    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.140279 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.140218    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/471de83b-f63f-4f68-90a4-fe119a1be894-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.140279 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.140258    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.140405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.140295    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.140664 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.140639    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfnmb\" (UniqueName: \"kubernetes.io/projected/471de83b-f63f-4f68-90a4-fe119a1be894-kube-api-access-bfnmb\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.140821 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.140782    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.241786 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.241750    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.242014 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.241802    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/471de83b-f63f-4f68-90a4-fe119a1be894-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.242014 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.241839    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.242014 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.241871    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.242014 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.241931    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnmb\" (UniqueName: \"kubernetes.io/projected/471de83b-f63f-4f68-90a4-fe119a1be894-kube-api-access-bfnmb\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.242014 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.241981    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.242587 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.242560    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/471de83b-f63f-4f68-90a4-fe119a1be894-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.245182 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.245155    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.245300 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.245276    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.245341 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.245320    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.245394 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.245378    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.265643 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.265614    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfnmb\" (UniqueName: \"kubernetes.io/projected/471de83b-f63f-4f68-90a4-fe119a1be894-kube-api-access-bfnmb\") pod \"data-science-collector-collector-1\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.315749 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.315719    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:43.367950 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.367895    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:43.443613 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.443582    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/399ba7f1-b339-460f-bd23-10413c751dc8-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/399ba7f1-b339-460f-bd23-10413c751dc8-data-science-collector-ta-server-cert\") pod \"399ba7f1-b339-460f-bd23-10413c751dc8\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") "
Oct 30 18:46:43.443760 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.443627    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/399ba7f1-b339-460f-bd23-10413c751dc8-ta-internal\" (UniqueName: \"kubernetes.io/configmap/399ba7f1-b339-460f-bd23-10413c751dc8-ta-internal\") pod \"399ba7f1-b339-460f-bd23-10413c751dc8\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") "
Oct 30 18:46:43.443760 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.443660    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/399ba7f1-b339-460f-bd23-10413c751dc8-kube-api-access-m7lmb\" (UniqueName: \"kubernetes.io/projected/399ba7f1-b339-460f-bd23-10413c751dc8-kube-api-access-m7lmb\") pod \"399ba7f1-b339-460f-bd23-10413c751dc8\" (UID: \"399ba7f1-b339-460f-bd23-10413c751dc8\") "
Oct 30 18:46:43.444113 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.444080    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/399ba7f1-b339-460f-bd23-10413c751dc8-ta-internal" pod "399ba7f1-b339-460f-bd23-10413c751dc8" (UID: "399ba7f1-b339-460f-bd23-10413c751dc8"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:43.445940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.445877    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/399ba7f1-b339-460f-bd23-10413c751dc8-kube-api-access-m7lmb" pod "399ba7f1-b339-460f-bd23-10413c751dc8" (UID: "399ba7f1-b339-460f-bd23-10413c751dc8"). InnerVolumeSpecName "kube-api-access-m7lmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:43.445940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.445882    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/399ba7f1-b339-460f-bd23-10413c751dc8-data-science-collector-ta-server-cert" pod "399ba7f1-b339-460f-bd23-10413c751dc8" (UID: "399ba7f1-b339-460f-bd23-10413c751dc8"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:43.544705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.544677    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/399ba7f1-b339-460f-bd23-10413c751dc8-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:43.544705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.544705    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/399ba7f1-b339-460f-bd23-10413c751dc8-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:43.544859 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.544716    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7lmb\" (UniqueName: \"kubernetes.io/projected/399ba7f1-b339-460f-bd23-10413c751dc8-kube-api-access-m7lmb\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:43.666757 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.666724    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:43.670427 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:43.670394    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471de83b_f63f_4f68_90a4_fe119a1be894.slice/crio-3f74b787dde8dd44c42220de9514258088c5ff282799ed5d3237c7c5bc860d24 WatchSource:0}: Error finding container 3f74b787dde8dd44c42220de9514258088c5ff282799ed5d3237c7c5bc860d24: Status 404 returned error can't find the container with id 3f74b787dde8dd44c42220de9514258088c5ff282799ed5d3237c7c5bc860d24
Oct 30 18:46:43.673074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.673043    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:43.673161 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.673099    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:43.968242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.967843    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"471de83b-f63f-4f68-90a4-fe119a1be894","Type":"ContainerStarted","Data":"686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7"}
Oct 30 18:46:43.968242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.967929    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"471de83b-f63f-4f68-90a4-fe119a1be894","Type":"ContainerStarted","Data":"3f74b787dde8dd44c42220de9514258088c5ff282799ed5d3237c7c5bc860d24"}
Oct 30 18:46:43.969260 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.969234    5746 generic.go:356] "Generic (PLEG): container finished" podID="399ba7f1-b339-460f-bd23-10413c751dc8" containerID="71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338" exitCode=0
Oct 30 18:46:43.969364 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.969264    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp" event={"ID":"399ba7f1-b339-460f-bd23-10413c751dc8","Type":"ContainerDied","Data":"71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338"}
Oct 30 18:46:43.969364 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.969297    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp" event={"ID":"399ba7f1-b339-460f-bd23-10413c751dc8","Type":"ContainerDied","Data":"d4dc6aa7aa4fd0cd1dea65c2a87a47506d5e01176b0748ab373b9340b9284872"}
Oct 30 18:46:43.969364 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.969318    5746 scope.go:122] "RemoveContainer" containerID="71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338"
Oct 30 18:46:43.969364 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.969270    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"
Oct 30 18:46:43.977876 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.977858    5746 scope.go:122] "RemoveContainer" containerID="71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338"
Oct 30 18:46:43.978198 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:43.978153    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338\": container with ID starting with 71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338 not found: ID does not exist" containerID="71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338"
Oct 30 18:46:43.978198 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:43.978187    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338"} err="failed to get container status \"71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338\": rpc error: code = NotFound desc = could not find container \"71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338\": container with ID starting with 71c3c0a5d3a6da19f1be5f3cca84db96314da48d823cc24001f9642880704338 not found: ID does not exist"
Oct 30 18:46:44.008980 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:44.008950    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4181de0-1237-40ab-bdfc-e76837eecfec" path="/var/lib/kubelet/pods/d4181de0-1237-40ab-bdfc-e76837eecfec/volumes"
Oct 30 18:46:44.009964 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:44.009859    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-1" podStartSLOduration=1.009841623 podStartE2EDuration="1.009841623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:44.009287721 +0000 UTC m=+2974.176105867" watchObservedRunningTime="2026-10-30 18:46:44.009841623 +0000 UTC m=+2974.176659754"
Oct 30 18:46:44.044131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:44.044097    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"]
Oct 30 18:46:44.049129 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:44.049100    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5f6bb744cd-7fjtp"]
Oct 30 18:46:46.007779 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:46.007741    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="399ba7f1-b339-460f-bd23-10413c751dc8" path="/var/lib/kubelet/pods/399ba7f1-b339-460f-bd23-10413c751dc8/volumes"
Oct 30 18:46:58.401176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.401140    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:58.401757 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.401367    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-1" podUID="471de83b-f63f-4f68-90a4-fe119a1be894" containerName="otc-container" containerID="cri-o://686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7" gracePeriod=30
Oct 30 18:46:58.633449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.633418    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"]
Oct 30 18:46:58.633768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.633752    5746 cpu_manager.go:414] "containerMap: removing container" podUID="399ba7f1-b339-460f-bd23-10413c751dc8" containerName="ta-container"
Oct 30 18:46:58.633768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.633768    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="399ba7f1-b339-460f-bd23-10413c751dc8" containerName="ta-container"
Oct 30 18:46:58.633973 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.633828    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="399ba7f1-b339-460f-bd23-10413c751dc8" containerName="ta-container"
Oct 30 18:46:58.637225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.637200    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.649719 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.649695    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:58.657325 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.657296    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"]
Oct 30 18:46:58.774726 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.774691    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-data-science-collector-ta-client-cert\") pod \"471de83b-f63f-4f68-90a4-fe119a1be894\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") "
Oct 30 18:46:58.774886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.774775    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-monitoring-tls-certs\") pod \"471de83b-f63f-4f68-90a4-fe119a1be894\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") "
Oct 30 18:46:58.774886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.774844    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/471de83b-f63f-4f68-90a4-fe119a1be894-otc-internal\" (UniqueName: \"kubernetes.io/configmap/471de83b-f63f-4f68-90a4-fe119a1be894-otc-internal\") pod \"471de83b-f63f-4f68-90a4-fe119a1be894\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") "
Oct 30 18:46:58.774886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.774876    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/471de83b-f63f-4f68-90a4-fe119a1be894-kube-api-access-bfnmb\" (UniqueName: \"kubernetes.io/projected/471de83b-f63f-4f68-90a4-fe119a1be894-kube-api-access-bfnmb\") pod \"471de83b-f63f-4f68-90a4-fe119a1be894\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") "
Oct 30 18:46:58.775090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.774938    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-tls-certs\") pod \"471de83b-f63f-4f68-90a4-fe119a1be894\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") "
Oct 30 18:46:58.775090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.774982    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-otlp-tls-certs\") pod \"471de83b-f63f-4f68-90a4-fe119a1be894\" (UID: \"471de83b-f63f-4f68-90a4-fe119a1be894\") "
Oct 30 18:46:58.775193 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.775129    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/5efe9f01-ae93-4d87-96d2-5930620853b6-ta-internal\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.775252 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.775197    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c4fk\" (UniqueName: \"kubernetes.io/projected/5efe9f01-ae93-4d87-96d2-5930620853b6-kube-api-access-2c4fk\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.775381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.775271    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5efe9f01-ae93-4d87-96d2-5930620853b6-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.775381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.775297    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/471de83b-f63f-4f68-90a4-fe119a1be894-otc-internal" pod "471de83b-f63f-4f68-90a4-fe119a1be894" (UID: "471de83b-f63f-4f68-90a4-fe119a1be894"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:58.777231 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.777199    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/471de83b-f63f-4f68-90a4-fe119a1be894-kube-api-access-bfnmb" pod "471de83b-f63f-4f68-90a4-fe119a1be894" (UID: "471de83b-f63f-4f68-90a4-fe119a1be894"). InnerVolumeSpecName "kube-api-access-bfnmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:58.777340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.777293    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-data-science-collector-ta-client-cert" pod "471de83b-f63f-4f68-90a4-fe119a1be894" (UID: "471de83b-f63f-4f68-90a4-fe119a1be894"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:58.777340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.777320    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-tls-certs" pod "471de83b-f63f-4f68-90a4-fe119a1be894" (UID: "471de83b-f63f-4f68-90a4-fe119a1be894"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:58.777700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.777679    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-otlp-tls-certs" pod "471de83b-f63f-4f68-90a4-fe119a1be894" (UID: "471de83b-f63f-4f68-90a4-fe119a1be894"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:58.777766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.777703    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-monitoring-tls-certs" pod "471de83b-f63f-4f68-90a4-fe119a1be894" (UID: "471de83b-f63f-4f68-90a4-fe119a1be894"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:58.876465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876425    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c4fk\" (UniqueName: \"kubernetes.io/projected/5efe9f01-ae93-4d87-96d2-5930620853b6-kube-api-access-2c4fk\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.876657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876508    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5efe9f01-ae93-4d87-96d2-5930620853b6-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.876657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876583    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/5efe9f01-ae93-4d87-96d2-5930620853b6-ta-internal\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.876657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876651    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:58.876820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876670    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/471de83b-f63f-4f68-90a4-fe119a1be894-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:58.876820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876685    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfnmb\" (UniqueName: \"kubernetes.io/projected/471de83b-f63f-4f68-90a4-fe119a1be894-kube-api-access-bfnmb\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:58.876820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876699    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:58.876820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876715    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:58.876820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.876730    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/471de83b-f63f-4f68-90a4-fe119a1be894-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:46:58.877278 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.877258    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/5efe9f01-ae93-4d87-96d2-5930620853b6-ta-internal\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.879199 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.879176    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5efe9f01-ae93-4d87-96d2-5930620853b6-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.899419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.899389    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c4fk\" (UniqueName: \"kubernetes.io/projected/5efe9f01-ae93-4d87-96d2-5930620853b6-kube-api-access-2c4fk\") pod \"data-science-collector-targetallocator-6c9c66c8d4-8nqb5\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:58.958347 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:58.958256    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:46:59.033179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.033143    5746 generic.go:356] "Generic (PLEG): container finished" podID="471de83b-f63f-4f68-90a4-fe119a1be894" containerID="686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7" exitCode=0
Oct 30 18:46:59.033353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.033302    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"471de83b-f63f-4f68-90a4-fe119a1be894","Type":"ContainerDied","Data":"686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7"}
Oct 30 18:46:59.033420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.033357    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"471de83b-f63f-4f68-90a4-fe119a1be894","Type":"ContainerDied","Data":"3f74b787dde8dd44c42220de9514258088c5ff282799ed5d3237c7c5bc860d24"}
Oct 30 18:46:59.033420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.033388    5746 scope.go:122] "RemoveContainer" containerID="686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7"
Oct 30 18:46:59.033578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.033561    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.043128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.043110    5746 scope.go:122] "RemoveContainer" containerID="686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7"
Oct 30 18:46:59.043451 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:46:59.043427    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7\": container with ID starting with 686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7 not found: ID does not exist" containerID="686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7"
Oct 30 18:46:59.043537 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.043464    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7"} err="failed to get container status \"686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7\": rpc error: code = NotFound desc = could not find container \"686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7\": container with ID starting with 686a701c2782e3b5c49dc465b1c65b3fa64c85172c76171fe67484c5456456c7 not found: ID does not exist"
Oct 30 18:46:59.084089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.084057    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:59.088091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.088064    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:59.125350 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.125300    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:59.125605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.125591    5746 cpu_manager.go:414] "containerMap: removing container" podUID="471de83b-f63f-4f68-90a4-fe119a1be894" containerName="otc-container"
Oct 30 18:46:59.125662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.125605    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="471de83b-f63f-4f68-90a4-fe119a1be894" containerName="otc-container"
Oct 30 18:46:59.125698 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.125675    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="471de83b-f63f-4f68-90a4-fe119a1be894" containerName="otc-container"
Oct 30 18:46:59.130083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.130062    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.144276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.144251    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-6d9d4583\""
Oct 30 18:46:59.144714 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.144694    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:46:59.144793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.144711    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:46:59.144793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.144717    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:46:59.144869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.144831    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:46:59.144929 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.144885    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsglb\""
Oct 30 18:46:59.149520 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.149499    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:59.180312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.180277    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.180312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.180313    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f48jk\" (UniqueName: \"kubernetes.io/projected/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-kube-api-access-f48jk\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.180562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.180347    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.180562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.180476    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.180562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.180515    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.180697 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.180615    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.256825 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.256743    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"]
Oct 30 18:46:59.260311 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:59.260283    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5efe9f01_ae93_4d87_96d2_5930620853b6.slice/crio-320e118ea2b944efa36da14610bbd8362278d528e56956b084d08899046334cd WatchSource:0}: Error finding container 320e118ea2b944efa36da14610bbd8362278d528e56956b084d08899046334cd: Status 404 returned error can't find the container with id 320e118ea2b944efa36da14610bbd8362278d528e56956b084d08899046334cd
Oct 30 18:46:59.262486 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.262454    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:59.262566 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.262516    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:59.281748 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.281727    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.281881 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.281769    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.281881 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.281823    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.281881 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.281859    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.282068 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.282020    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f48jk\" (UniqueName: \"kubernetes.io/projected/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-kube-api-access-f48jk\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.282128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.282082    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.282770 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.282735    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.284314 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.284287    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.284427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.284372    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.284427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.284400    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.284427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.284408    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.305112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.305089    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f48jk\" (UniqueName: \"kubernetes.io/projected/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-kube-api-access-f48jk\") pod \"data-science-collector-collector-1\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.441645 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.441607    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:46:59.740307 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.740277    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:46:59.744202 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:46:59.744172    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14cdfb7d_fbfa_46e9_8af3_f800de5777d2.slice/crio-cb46f07766c52f703fde73fe061063fb0998033f8c5eb465d7a0a4cec2ce1c53 WatchSource:0}: Error finding container cb46f07766c52f703fde73fe061063fb0998033f8c5eb465d7a0a4cec2ce1c53: Status 404 returned error can't find the container with id cb46f07766c52f703fde73fe061063fb0998033f8c5eb465d7a0a4cec2ce1c53
Oct 30 18:46:59.746551 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.746515    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:46:59.746663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:46:59.746593    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:47:00.008737 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.008666    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="471de83b-f63f-4f68-90a4-fe119a1be894" path="/var/lib/kubelet/pods/471de83b-f63f-4f68-90a4-fe119a1be894/volumes"
Oct 30 18:47:00.039231 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.039195    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"14cdfb7d-fbfa-46e9-8af3-f800de5777d2","Type":"ContainerStarted","Data":"716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64"}
Oct 30 18:47:00.039404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.039264    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"14cdfb7d-fbfa-46e9-8af3-f800de5777d2","Type":"ContainerStarted","Data":"cb46f07766c52f703fde73fe061063fb0998033f8c5eb465d7a0a4cec2ce1c53"}
Oct 30 18:47:00.040784 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.040747    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5" event={"ID":"5efe9f01-ae93-4d87-96d2-5930620853b6","Type":"ContainerStarted","Data":"412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce"}
Oct 30 18:47:00.040784 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.040788    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5" event={"ID":"5efe9f01-ae93-4d87-96d2-5930620853b6","Type":"ContainerStarted","Data":"320e118ea2b944efa36da14610bbd8362278d528e56956b084d08899046334cd"}
Oct 30 18:47:00.041008 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.040807    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:47:00.042405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.042384    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:47:00.081150 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.081101    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-1" podStartSLOduration=1.081086932 podStartE2EDuration="1.081086932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:00.080610684 +0000 UTC m=+2990.247428810" watchObservedRunningTime="2026-10-30 18:47:00.081086932 +0000 UTC m=+2990.247905059"
Oct 30 18:47:00.121600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.121525    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5" podStartSLOduration=2.121503752 podStartE2EDuration="2.121503752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:00.120402397 +0000 UTC m=+2990.287220550" watchObservedRunningTime="2026-10-30 18:47:00.121503752 +0000 UTC m=+2990.288321877"
Oct 30 18:47:00.153565 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.153522    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"]
Oct 30 18:47:00.154407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.154229    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w" podUID="5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" containerName="ta-container" containerID="cri-o://3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2" gracePeriod=30
Oct 30 18:47:00.389793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.389769    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:47:00.493986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.493957    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-ta-internal\" (UniqueName: \"kubernetes.io/configmap/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-ta-internal\") pod \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") "
Oct 30 18:47:00.494438 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.494014    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-data-science-collector-ta-server-cert\") pod \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") "
Oct 30 18:47:00.494438 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.494056    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-kube-api-access-zpsn4\" (UniqueName: \"kubernetes.io/projected/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-kube-api-access-zpsn4\") pod \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\" (UID: \"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9\") "
Oct 30 18:47:00.494438 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.494295    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-ta-internal" pod "5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" (UID: "5b2af657-6a9e-47ec-8386-4f79c5e6fcf9"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:00.496211 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.496187    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-data-science-collector-ta-server-cert" pod "5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" (UID: "5b2af657-6a9e-47ec-8386-4f79c5e6fcf9"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:00.496295 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.496280    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-kube-api-access-zpsn4" pod "5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" (UID: "5b2af657-6a9e-47ec-8386-4f79c5e6fcf9"). InnerVolumeSpecName "kube-api-access-zpsn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:00.594776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.594679    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpsn4\" (UniqueName: \"kubernetes.io/projected/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-kube-api-access-zpsn4\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:00.594776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.594723    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:00.594776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:00.594736    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:01.047183 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.047153    5746 generic.go:356] "Generic (PLEG): container finished" podID="5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" containerID="3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2" exitCode=0
Oct 30 18:47:01.047376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.047200    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"
Oct 30 18:47:01.047376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.047243    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w" event={"ID":"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9","Type":"ContainerDied","Data":"3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2"}
Oct 30 18:47:01.047376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.047289    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w" event={"ID":"5b2af657-6a9e-47ec-8386-4f79c5e6fcf9","Type":"ContainerDied","Data":"8859e2a4066d2b97fe2b70728a211eb39c879ee7577819fd9e8c0194b94c27d2"}
Oct 30 18:47:01.047376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.047313    5746 scope.go:122] "RemoveContainer" containerID="3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2"
Oct 30 18:47:01.056105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.056089    5746 scope.go:122] "RemoveContainer" containerID="3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2"
Oct 30 18:47:01.056365 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:01.056345    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2\": container with ID starting with 3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2 not found: ID does not exist" containerID="3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2"
Oct 30 18:47:01.056412 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.056375    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2"} err="failed to get container status \"3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2\": rpc error: code = NotFound desc = could not find container \"3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2\": container with ID starting with 3ff865447c54ee28f5717595a1f342a1077ffd0c7a16f01b672db0adf858fbe2 not found: ID does not exist"
Oct 30 18:47:01.096579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.096551    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"]
Oct 30 18:47:01.100851 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:01.100828    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-f9845c957-btr6w"]
Oct 30 18:47:02.008630 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:02.008601    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" path="/var/lib/kubelet/pods/5b2af657-6a9e-47ec-8386-4f79c5e6fcf9/volumes"
Oct 30 18:47:12.790478 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:12.790432    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:47:15.543994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.543952    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:15.544491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.544207    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-1" podUID="14cdfb7d-fbfa-46e9-8af3-f800de5777d2" containerName="otc-container" containerID="cri-o://716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64" gracePeriod=30
Oct 30 18:47:15.774401 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.774369    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"]
Oct 30 18:47:15.774676 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.774662    5746 cpu_manager.go:414] "containerMap: removing container" podUID="5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" containerName="ta-container"
Oct 30 18:47:15.774731 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.774677    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" containerName="ta-container"
Oct 30 18:47:15.774766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.774742    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b2af657-6a9e-47ec-8386-4f79c5e6fcf9" containerName="ta-container"
Oct 30 18:47:15.779115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.779097    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.797623 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.797564    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"]
Oct 30 18:47:15.811441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.811419    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-ta-internal\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.811540 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.811450    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw9ff\" (UniqueName: \"kubernetes.io/projected/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-kube-api-access-fw9ff\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.811540 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.811513    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.912758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.912728    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-ta-internal\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.912758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.912762    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw9ff\" (UniqueName: \"kubernetes.io/projected/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-kube-api-access-fw9ff\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.912971 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.912784    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.913388 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.913369    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-ta-internal\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.915351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.915326    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:15.935556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:15.935523    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw9ff\" (UniqueName: \"kubernetes.io/projected/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-kube-api-access-fw9ff\") pod \"data-science-collector-targetallocator-5c767fcc7f-9w8pn\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:16.089709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.089635    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:16.389940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.389888    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"]
Oct 30 18:47:16.392720 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:47:16.392692    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfde1e29e_0b9f_4713_94fa_c6edc979ccb7.slice/crio-a173241c55cb95beb162c50dedb84660515882cb0b74f5568419dffab3e95c35 WatchSource:0}: Error finding container a173241c55cb95beb162c50dedb84660515882cb0b74f5568419dffab3e95c35: Status 404 returned error can't find the container with id a173241c55cb95beb162c50dedb84660515882cb0b74f5568419dffab3e95c35
Oct 30 18:47:16.395057 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.395021    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:47:16.395178 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.395089    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:47:16.783162 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.783137    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:16.820178 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.820151    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otlp-tls-certs\") pod \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") "
Oct 30 18:47:16.820321 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.820187    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-tls-certs\") pod \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") "
Oct 30 18:47:16.820321 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.820218    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otc-internal\" (UniqueName: \"kubernetes.io/configmap/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otc-internal\") pod \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") "
Oct 30 18:47:16.820441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.820336    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-monitoring-tls-certs\") pod \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") "
Oct 30 18:47:16.820441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.820379    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-kube-api-access-f48jk\" (UniqueName: \"kubernetes.io/projected/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-kube-api-access-f48jk\") pod \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") "
Oct 30 18:47:16.820551 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.820443    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-data-science-collector-ta-client-cert\") pod \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\" (UID: \"14cdfb7d-fbfa-46e9-8af3-f800de5777d2\") "
Oct 30 18:47:16.820648 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.820624    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otc-internal" pod "14cdfb7d-fbfa-46e9-8af3-f800de5777d2" (UID: "14cdfb7d-fbfa-46e9-8af3-f800de5777d2"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:16.820810 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.820779    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:16.822577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.822545    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-data-science-collector-ta-client-cert" pod "14cdfb7d-fbfa-46e9-8af3-f800de5777d2" (UID: "14cdfb7d-fbfa-46e9-8af3-f800de5777d2"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:16.822684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.822605    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-tls-certs" pod "14cdfb7d-fbfa-46e9-8af3-f800de5777d2" (UID: "14cdfb7d-fbfa-46e9-8af3-f800de5777d2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:16.822684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.822626    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otlp-tls-certs" pod "14cdfb7d-fbfa-46e9-8af3-f800de5777d2" (UID: "14cdfb7d-fbfa-46e9-8af3-f800de5777d2"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:16.823044 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.823028    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-kube-api-access-f48jk" pod "14cdfb7d-fbfa-46e9-8af3-f800de5777d2" (UID: "14cdfb7d-fbfa-46e9-8af3-f800de5777d2"). InnerVolumeSpecName "kube-api-access-f48jk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:16.823469 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.823456    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-monitoring-tls-certs" pod "14cdfb7d-fbfa-46e9-8af3-f800de5777d2" (UID: "14cdfb7d-fbfa-46e9-8af3-f800de5777d2"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:16.921720 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.921688    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:16.921720 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.921719    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f48jk\" (UniqueName: \"kubernetes.io/projected/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-kube-api-access-f48jk\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:16.921937 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.921735    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:16.921937 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.921744    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:16.921937 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:16.921754    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14cdfb7d-fbfa-46e9-8af3-f800de5777d2-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:17.116185 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.116151    5746 generic.go:356] "Generic (PLEG): container finished" podID="14cdfb7d-fbfa-46e9-8af3-f800de5777d2" containerID="716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64" exitCode=0
Oct 30 18:47:17.116381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.116196    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.116381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.116235    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"14cdfb7d-fbfa-46e9-8af3-f800de5777d2","Type":"ContainerDied","Data":"716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64"}
Oct 30 18:47:17.116381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.116296    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"14cdfb7d-fbfa-46e9-8af3-f800de5777d2","Type":"ContainerDied","Data":"cb46f07766c52f703fde73fe061063fb0998033f8c5eb465d7a0a4cec2ce1c53"}
Oct 30 18:47:17.116381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.116325    5746 scope.go:122] "RemoveContainer" containerID="716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64"
Oct 30 18:47:17.118016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.117992    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn" event={"ID":"fde1e29e-0b9f-4713-94fa-c6edc979ccb7","Type":"ContainerStarted","Data":"f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36"}
Oct 30 18:47:17.118158 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.118027    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn" event={"ID":"fde1e29e-0b9f-4713-94fa-c6edc979ccb7","Type":"ContainerStarted","Data":"a173241c55cb95beb162c50dedb84660515882cb0b74f5568419dffab3e95c35"}
Oct 30 18:47:17.118158 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.118043    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:17.120172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.120152    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:17.125673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.125656    5746 scope.go:122] "RemoveContainer" containerID="716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64"
Oct 30 18:47:17.125953 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:17.125932    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64\": container with ID starting with 716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64 not found: ID does not exist" containerID="716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64"
Oct 30 18:47:17.126001 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.125963    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64"} err="failed to get container status \"716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64\": rpc error: code = NotFound desc = could not find container \"716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64\": container with ID starting with 716968bc258ad6bd0a961f05ce219895c3fd87949f4e70a6aa87088c34466b64 not found: ID does not exist"
Oct 30 18:47:17.172947 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.172853    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:17.178773 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.178750    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:17.220714 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.220678    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:17.221000 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.220987    5746 cpu_manager.go:414] "containerMap: removing container" podUID="14cdfb7d-fbfa-46e9-8af3-f800de5777d2" containerName="otc-container"
Oct 30 18:47:17.221064 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.221002    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14cdfb7d-fbfa-46e9-8af3-f800de5777d2" containerName="otc-container"
Oct 30 18:47:17.221099 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.221063    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="14cdfb7d-fbfa-46e9-8af3-f800de5777d2" containerName="otc-container"
Oct 30 18:47:17.225156 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.225136    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.232233 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.232180    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn" podStartSLOduration=2.232164785 podStartE2EDuration="2.232164785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:17.231873676 +0000 UTC m=+3007.398691828" watchObservedRunningTime="2026-10-30 18:47:17.232164785 +0000 UTC m=+3007.398982906"
Oct 30 18:47:17.241391 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.241372    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-client-cert\""
Oct 30 18:47:17.241984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.241964    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-tls\""
Oct 30 18:47:17.241984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.241979    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-headless-tls\""
Oct 30 18:47:17.242112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.241969    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-dockercfg-hsglb\""
Oct 30 18:47:17.242112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.242057    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-monitoring-tls\""
Oct 30 18:47:17.254374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.254313    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-collector-a0f9182a\""
Oct 30 18:47:17.254729 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.254699    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:17.279605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.279577    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"]
Oct 30 18:47:17.279828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.279791    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5" podUID="5efe9f01-ae93-4d87-96d2-5930620853b6" containerName="ta-container" containerID="cri-o://412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce" gracePeriod=30
Oct 30 18:47:17.324610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.324573    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n62dz\" (UniqueName: \"kubernetes.io/projected/07232107-7667-4721-abe8-6b5466d95f6c-kube-api-access-n62dz\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.324750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.324624    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.324750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.324720    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.324884 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.324785    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/07232107-7667-4721-abe8-6b5466d95f6c-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.324884 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.324836    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.324995 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.324890    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.425786 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.425706    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.425786 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.425757    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.426027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.425803    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n62dz\" (UniqueName: \"kubernetes.io/projected/07232107-7667-4721-abe8-6b5466d95f6c-kube-api-access-n62dz\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.426027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.425833    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.426027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.425866    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.426027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.425945    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/07232107-7667-4721-abe8-6b5466d95f6c-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.426716 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.426685    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/07232107-7667-4721-abe8-6b5466d95f6c-otc-internal\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.428876 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.428851    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.428985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.428857    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-monitoring-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.429156 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.429139    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-otlp-tls-certs\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.429571 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.429551    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-data-science-collector-ta-client-cert\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.449108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.449083    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n62dz\" (UniqueName: \"kubernetes.io/projected/07232107-7667-4721-abe8-6b5466d95f6c-kube-api-access-n62dz\") pod \"data-science-collector-collector-1\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") " pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.514078 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.514054    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:47:17.538550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.538528    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:17.628206 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.628179    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5efe9f01-ae93-4d87-96d2-5930620853b6-ta-internal\" (UniqueName: \"kubernetes.io/configmap/5efe9f01-ae93-4d87-96d2-5930620853b6-ta-internal\") pod \"5efe9f01-ae93-4d87-96d2-5930620853b6\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") "
Oct 30 18:47:17.628356 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.628221    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5efe9f01-ae93-4d87-96d2-5930620853b6-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5efe9f01-ae93-4d87-96d2-5930620853b6-data-science-collector-ta-server-cert\") pod \"5efe9f01-ae93-4d87-96d2-5930620853b6\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") "
Oct 30 18:47:17.628400 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.628357    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5efe9f01-ae93-4d87-96d2-5930620853b6-kube-api-access-2c4fk\" (UniqueName: \"kubernetes.io/projected/5efe9f01-ae93-4d87-96d2-5930620853b6-kube-api-access-2c4fk\") pod \"5efe9f01-ae93-4d87-96d2-5930620853b6\" (UID: \"5efe9f01-ae93-4d87-96d2-5930620853b6\") "
Oct 30 18:47:17.628563 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.628540    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5efe9f01-ae93-4d87-96d2-5930620853b6-ta-internal" pod "5efe9f01-ae93-4d87-96d2-5930620853b6" (UID: "5efe9f01-ae93-4d87-96d2-5930620853b6"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:17.628655 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.628614    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/5efe9f01-ae93-4d87-96d2-5930620853b6-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:17.630296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.630276    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5efe9f01-ae93-4d87-96d2-5930620853b6-data-science-collector-ta-server-cert" pod "5efe9f01-ae93-4d87-96d2-5930620853b6" (UID: "5efe9f01-ae93-4d87-96d2-5930620853b6"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:17.630478 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.630457    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5efe9f01-ae93-4d87-96d2-5930620853b6-kube-api-access-2c4fk" pod "5efe9f01-ae93-4d87-96d2-5930620853b6" (UID: "5efe9f01-ae93-4d87-96d2-5930620853b6"). InnerVolumeSpecName "kube-api-access-2c4fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:17.729513 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.729435    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/5efe9f01-ae93-4d87-96d2-5930620853b6-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:17.729513 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.729457    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c4fk\" (UniqueName: \"kubernetes.io/projected/5efe9f01-ae93-4d87-96d2-5930620853b6-kube-api-access-2c4fk\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:17.837257 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.837223    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:17.840375 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:47:17.840352    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07232107_7667_4721_abe8_6b5466d95f6c.slice/crio-82d50c3e798fa3b30c7250e49f5ff9d84d0ef827f6b1ca83c884435caf1249cc WatchSource:0}: Error finding container 82d50c3e798fa3b30c7250e49f5ff9d84d0ef827f6b1ca83c884435caf1249cc: Status 404 returned error can't find the container with id 82d50c3e798fa3b30c7250e49f5ff9d84d0ef827f6b1ca83c884435caf1249cc
Oct 30 18:47:17.842498 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.842467    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:47:17.842550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:17.842527    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:47:18.010268 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.010191    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14cdfb7d-fbfa-46e9-8af3-f800de5777d2" path="/var/lib/kubelet/pods/14cdfb7d-fbfa-46e9-8af3-f800de5777d2/volumes"
Oct 30 18:47:18.126679 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.126643    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"07232107-7667-4721-abe8-6b5466d95f6c","Type":"ContainerStarted","Data":"3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0"}
Oct 30 18:47:18.126876 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.126706    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"07232107-7667-4721-abe8-6b5466d95f6c","Type":"ContainerStarted","Data":"82d50c3e798fa3b30c7250e49f5ff9d84d0ef827f6b1ca83c884435caf1249cc"}
Oct 30 18:47:18.129081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.129047    5746 generic.go:356] "Generic (PLEG): container finished" podID="5efe9f01-ae93-4d87-96d2-5930620853b6" containerID="412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce" exitCode=0
Oct 30 18:47:18.129207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.129083    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"
Oct 30 18:47:18.129207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.129129    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5" event={"ID":"5efe9f01-ae93-4d87-96d2-5930620853b6","Type":"ContainerDied","Data":"412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce"}
Oct 30 18:47:18.129207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.129161    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5" event={"ID":"5efe9f01-ae93-4d87-96d2-5930620853b6","Type":"ContainerDied","Data":"320e118ea2b944efa36da14610bbd8362278d528e56956b084d08899046334cd"}
Oct 30 18:47:18.129207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.129177    5746 scope.go:122] "RemoveContainer" containerID="412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce"
Oct 30 18:47:18.137396 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.137379    5746 scope.go:122] "RemoveContainer" containerID="412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce"
Oct 30 18:47:18.137676 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:18.137650    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce\": container with ID starting with 412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce not found: ID does not exist" containerID="412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce"
Oct 30 18:47:18.137762 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.137673    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce"} err="failed to get container status \"412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce\": rpc error: code = NotFound desc = could not find container \"412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce\": container with ID starting with 412ca169d859699ecc8cd58b8e4ff98ac61cd209672602b3ab48649c2df259ce not found: ID does not exist"
Oct 30 18:47:18.170193 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.170139    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-collector-1" podStartSLOduration=1.170125071 podStartE2EDuration="1.170125071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:18.169152254 +0000 UTC m=+3008.335970400" watchObservedRunningTime="2026-10-30 18:47:18.170125071 +0000 UTC m=+3008.336943200"
Oct 30 18:47:18.204033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.204001    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"]
Oct 30 18:47:18.206633 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:18.206608    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c9c66c8d4-8nqb5"]
Oct 30 18:47:20.009857 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:20.009822    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5efe9f01-ae93-4d87-96d2-5930620853b6" path="/var/lib/kubelet/pods/5efe9f01-ae93-4d87-96d2-5930620853b6/volumes"
Oct 30 18:47:24.919569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:24.919532    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"]
Oct 30 18:47:24.922033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:24.920058    5746 cpu_manager.go:414] "containerMap: removing container" podUID="5efe9f01-ae93-4d87-96d2-5930620853b6" containerName="ta-container"
Oct 30 18:47:24.922033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:24.920080    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5efe9f01-ae93-4d87-96d2-5930620853b6" containerName="ta-container"
Oct 30 18:47:24.922033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:24.920173    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="5efe9f01-ae93-4d87-96d2-5930620853b6" containerName="ta-container"
Oct 30 18:47:24.926482 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:24.926460    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"
Oct 30 18:47:24.935123 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:24.934872    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"]
Oct 30 18:47:24.940888 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:24.940860    5746 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7cx2x], unattached volumes=[], failed to process volumes=[kube-api-access-7cx2x]: context canceled" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7" podUID="a0b04cd0-b2e9-4335-ab23-2acc4bd29999"
Oct 30 18:47:24.989698 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:24.989663    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"]
Oct 30 18:47:24.993798 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:24.993761    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cx2x\" (UniqueName: \"kubernetes.io/projected/a0b04cd0-b2e9-4335-ab23-2acc4bd29999-kube-api-access-7cx2x\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7\" (UID: \"a0b04cd0-b2e9-4335-ab23-2acc4bd29999\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"
Oct 30 18:47:25.094540 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.094498    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cx2x\" (UniqueName: \"kubernetes.io/projected/a0b04cd0-b2e9-4335-ab23-2acc4bd29999-kube-api-access-7cx2x\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7\" (UID: \"a0b04cd0-b2e9-4335-ab23-2acc4bd29999\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"
Oct 30 18:47:25.118149 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.118120    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cx2x\" (UniqueName: \"kubernetes.io/projected/a0b04cd0-b2e9-4335-ab23-2acc4bd29999-kube-api-access-7cx2x\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7\" (UID: \"a0b04cd0-b2e9-4335-ab23-2acc4bd29999\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"
Oct 30 18:47:25.159612 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.159581    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"
Oct 30 18:47:25.166985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.166960    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"
Oct 30 18:47:25.296980 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.296875    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0b04cd0-b2e9-4335-ab23-2acc4bd29999-kube-api-access-7cx2x\" (UniqueName: \"kubernetes.io/projected/a0b04cd0-b2e9-4335-ab23-2acc4bd29999-kube-api-access-7cx2x\") pod \"a0b04cd0-b2e9-4335-ab23-2acc4bd29999\" (UID: \"a0b04cd0-b2e9-4335-ab23-2acc4bd29999\") "
Oct 30 18:47:25.299296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.299264    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0b04cd0-b2e9-4335-ab23-2acc4bd29999-kube-api-access-7cx2x" pod "a0b04cd0-b2e9-4335-ab23-2acc4bd29999" (UID: "a0b04cd0-b2e9-4335-ab23-2acc4bd29999"). InnerVolumeSpecName "kube-api-access-7cx2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:25.398410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.398381    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cx2x\" (UniqueName: \"kubernetes.io/projected/a0b04cd0-b2e9-4335-ab23-2acc4bd29999-kube-api-access-7cx2x\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:25.641366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.641335    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:25.641593 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.641567    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-collector-1" podUID="07232107-7667-4721-abe8-6b5466d95f6c" containerName="otc-container" containerID="cri-o://3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0" gracePeriod=30
Oct 30 18:47:25.690398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.690360    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"]
Oct 30 18:47:25.690597 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.690574    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn" podUID="fde1e29e-0b9f-4713-94fa-c6edc979ccb7" containerName="ta-container" containerID="cri-o://f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36" gracePeriod=30
Oct 30 18:47:25.791811 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.791777    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:47:25.792093 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.792061    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="101ee503-0f76-4a6b-af62-35f62eff638f" containerName="perses" containerID="cri-o://3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87" gracePeriod=30
Oct 30 18:47:25.947275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.947247    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:25.952083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:25.952061    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:26.003500 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003467    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07232107-7667-4721-abe8-6b5466d95f6c-otc-internal\" (UniqueName: \"kubernetes.io/configmap/07232107-7667-4721-abe8-6b5466d95f6c-otc-internal\") pod \"07232107-7667-4721-abe8-6b5466d95f6c\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") "
Oct 30 18:47:26.003688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003534    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-tls-certs\") pod \"07232107-7667-4721-abe8-6b5466d95f6c\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") "
Oct 30 18:47:26.003688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003584    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-monitoring-tls-certs\") pod \"07232107-7667-4721-abe8-6b5466d95f6c\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") "
Oct 30 18:47:26.003688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003617    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-data-science-collector-ta-client-cert\") pod \"07232107-7667-4721-abe8-6b5466d95f6c\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") "
Oct 30 18:47:26.003863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003684    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-data-science-collector-ta-server-cert\") pod \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") "
Oct 30 18:47:26.003863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003752    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-otlp-tls-certs\") pod \"07232107-7667-4721-abe8-6b5466d95f6c\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") "
Oct 30 18:47:26.003863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003795    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07232107-7667-4721-abe8-6b5466d95f6c-kube-api-access-n62dz\" (UniqueName: \"kubernetes.io/projected/07232107-7667-4721-abe8-6b5466d95f6c-kube-api-access-n62dz\") pod \"07232107-7667-4721-abe8-6b5466d95f6c\" (UID: \"07232107-7667-4721-abe8-6b5466d95f6c\") "
Oct 30 18:47:26.004312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003937    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07232107-7667-4721-abe8-6b5466d95f6c-otc-internal" pod "07232107-7667-4721-abe8-6b5466d95f6c" (UID: "07232107-7667-4721-abe8-6b5466d95f6c"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:26.004312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.003894    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-kube-api-access-fw9ff\" (UniqueName: \"kubernetes.io/projected/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-kube-api-access-fw9ff\") pod \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") "
Oct 30 18:47:26.005317 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.004023    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-ta-internal\" (UniqueName: \"kubernetes.io/configmap/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-ta-internal\") pod \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\" (UID: \"fde1e29e-0b9f-4713-94fa-c6edc979ccb7\") "
Oct 30 18:47:26.005317 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.004872    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/07232107-7667-4721-abe8-6b5466d95f6c-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.005317 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.005276    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-ta-internal" pod "fde1e29e-0b9f-4713-94fa-c6edc979ccb7" (UID: "fde1e29e-0b9f-4713-94fa-c6edc979ccb7"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:26.007530 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.007499    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-otlp-tls-certs" pod "07232107-7667-4721-abe8-6b5466d95f6c" (UID: "07232107-7667-4721-abe8-6b5466d95f6c"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:26.009116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.009080    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07232107-7667-4721-abe8-6b5466d95f6c-kube-api-access-n62dz" pod "07232107-7667-4721-abe8-6b5466d95f6c" (UID: "07232107-7667-4721-abe8-6b5466d95f6c"). InnerVolumeSpecName "kube-api-access-n62dz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:26.010422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.009971    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-tls-certs" pod "07232107-7667-4721-abe8-6b5466d95f6c" (UID: "07232107-7667-4721-abe8-6b5466d95f6c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:26.010422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.010387    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-monitoring-tls-certs" pod "07232107-7667-4721-abe8-6b5466d95f6c" (UID: "07232107-7667-4721-abe8-6b5466d95f6c"). InnerVolumeSpecName "monitoring-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:26.011735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.011698    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-data-science-collector-ta-client-cert" pod "07232107-7667-4721-abe8-6b5466d95f6c" (UID: "07232107-7667-4721-abe8-6b5466d95f6c"). InnerVolumeSpecName "data-science-collector-ta-client-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:26.011854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.011825    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-data-science-collector-ta-server-cert" pod "fde1e29e-0b9f-4713-94fa-c6edc979ccb7" (UID: "fde1e29e-0b9f-4713-94fa-c6edc979ccb7"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:26.013473 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.013432    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-kube-api-access-fw9ff" pod "fde1e29e-0b9f-4713-94fa-c6edc979ccb7" (UID: "fde1e29e-0b9f-4713-94fa-c6edc979ccb7"). InnerVolumeSpecName "kube-api-access-fw9ff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:26.055654 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.055624    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:26.106010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.105977    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/101ee503-0f76-4a6b-af62-35f62eff638f-plugins\" (UniqueName: \"kubernetes.io/empty-dir/101ee503-0f76-4a6b-af62-35f62eff638f-plugins\") pod \"101ee503-0f76-4a6b-af62-35f62eff638f\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") "
Oct 30 18:47:26.106201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106031    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/101ee503-0f76-4a6b-af62-35f62eff638f-config\" (UniqueName: \"kubernetes.io/configmap/101ee503-0f76-4a6b-af62-35f62eff638f-config\") pod \"101ee503-0f76-4a6b-af62-35f62eff638f\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") "
Oct 30 18:47:26.106201 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106114    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/101ee503-0f76-4a6b-af62-35f62eff638f-kube-api-access-v9tmq\" (UniqueName: \"kubernetes.io/projected/101ee503-0f76-4a6b-af62-35f62eff638f-kube-api-access-v9tmq\") pod \"101ee503-0f76-4a6b-af62-35f62eff638f\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") "
Oct 30 18:47:26.106363 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106341    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"101ee503-0f76-4a6b-af62-35f62eff638f\" (UID: \"101ee503-0f76-4a6b-af62-35f62eff638f\") "
Oct 30 18:47:26.106419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106361    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/101ee503-0f76-4a6b-af62-35f62eff638f-config" pod "101ee503-0f76-4a6b-af62-35f62eff638f" (UID: "101ee503-0f76-4a6b-af62-35f62eff638f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:26.106586 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106568    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.106662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106591    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n62dz\" (UniqueName: \"kubernetes.io/projected/07232107-7667-4721-abe8-6b5466d95f6c-kube-api-access-n62dz\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.106662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106608    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw9ff\" (UniqueName: \"kubernetes.io/projected/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-kube-api-access-fw9ff\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.106662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106622    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.106662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106636    5746 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.106662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106649    5746 reconciler_common.go:299] "Volume detached for volume \"monitoring-tls-certs\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-monitoring-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.107104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106664    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-client-cert\" (UniqueName: \"kubernetes.io/secret/07232107-7667-4721-abe8-6b5466d95f6c-data-science-collector-ta-client-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.107104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106680    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/fde1e29e-0b9f-4713-94fa-c6edc979ccb7-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.107104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.106695    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/101ee503-0f76-4a6b-af62-35f62eff638f-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.109103 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.109063    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/101ee503-0f76-4a6b-af62-35f62eff638f-kube-api-access-v9tmq" pod "101ee503-0f76-4a6b-af62-35f62eff638f" (UID: "101ee503-0f76-4a6b-af62-35f62eff638f"). InnerVolumeSpecName "kube-api-access-v9tmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:26.109634 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.109593    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "101ee503-0f76-4a6b-af62-35f62eff638f" (UID: "101ee503-0f76-4a6b-af62-35f62eff638f"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:47:26.164766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.164696    5746 generic.go:356] "Generic (PLEG): container finished" podID="fde1e29e-0b9f-4713-94fa-c6edc979ccb7" containerID="f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36" exitCode=0
Oct 30 18:47:26.164766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.164741    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"
Oct 30 18:47:26.164998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.164776    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn" event={"ID":"fde1e29e-0b9f-4713-94fa-c6edc979ccb7","Type":"ContainerDied","Data":"f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36"}
Oct 30 18:47:26.164998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.164816    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn" event={"ID":"fde1e29e-0b9f-4713-94fa-c6edc979ccb7","Type":"ContainerDied","Data":"a173241c55cb95beb162c50dedb84660515882cb0b74f5568419dffab3e95c35"}
Oct 30 18:47:26.164998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.164837    5746 scope.go:122] "RemoveContainer" containerID="f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36"
Oct 30 18:47:26.166274 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.166249    5746 generic.go:356] "Generic (PLEG): container finished" podID="07232107-7667-4721-abe8-6b5466d95f6c" containerID="3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0" exitCode=0
Oct 30 18:47:26.166381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.166284    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"07232107-7667-4721-abe8-6b5466d95f6c","Type":"ContainerDied","Data":"3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0"}
Oct 30 18:47:26.166381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.166316    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-collector-1"
Oct 30 18:47:26.166381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.166335    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-collector-1" event={"ID":"07232107-7667-4721-abe8-6b5466d95f6c","Type":"ContainerDied","Data":"82d50c3e798fa3b30c7250e49f5ff9d84d0ef827f6b1ca83c884435caf1249cc"}
Oct 30 18:47:26.168175 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.168143    5746 generic.go:356] "Generic (PLEG): container finished" podID="101ee503-0f76-4a6b-af62-35f62eff638f" containerID="3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87" exitCode=0
Oct 30 18:47:26.168277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.168184    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:26.168277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.168218    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"101ee503-0f76-4a6b-af62-35f62eff638f","Type":"ContainerDied","Data":"3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87"}
Oct 30 18:47:26.168277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.168259    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"101ee503-0f76-4a6b-af62-35f62eff638f","Type":"ContainerDied","Data":"48db2fef86ae9568b8d93c81910a9cbf06611bb4e66b8c87543c5bbf08100581"}
Oct 30 18:47:26.168440 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.168295    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"
Oct 30 18:47:26.174797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.174777    5746 scope.go:122] "RemoveContainer" containerID="f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36"
Oct 30 18:47:26.175127 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:26.175100    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36\": container with ID starting with f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36 not found: ID does not exist" containerID="f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36"
Oct 30 18:47:26.175214 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.175138    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36"} err="failed to get container status \"f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36\": rpc error: code = NotFound desc = could not find container \"f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36\": container with ID starting with f17f07d3a99349e05331d303a10cbcefc4258500f8ac1dca0087f53e651b1b36 not found: ID does not exist"
Oct 30 18:47:26.175214 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.175156    5746 scope.go:122] "RemoveContainer" containerID="3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0"
Oct 30 18:47:26.182611 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.182594    5746 scope.go:122] "RemoveContainer" containerID="3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0"
Oct 30 18:47:26.182885 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:26.182866    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0\": container with ID starting with 3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0 not found: ID does not exist" containerID="3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0"
Oct 30 18:47:26.182961 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.182894    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0"} err="failed to get container status \"3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0\": rpc error: code = NotFound desc = could not find container \"3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0\": container with ID starting with 3ad5f179cfd2f242bc23b67e3299a3a9f9f413149425d8c66d9a6ab0efb4f7b0 not found: ID does not exist"
Oct 30 18:47:26.182961 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.182926    5746 scope.go:122] "RemoveContainer" containerID="3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87"
Oct 30 18:47:26.190073 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.190055    5746 scope.go:122] "RemoveContainer" containerID="3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87"
Oct 30 18:47:26.190326 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:26.190298    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87\": container with ID starting with 3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87 not found: ID does not exist" containerID="3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87"
Oct 30 18:47:26.190392 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.190330    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87"} err="failed to get container status \"3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87\": rpc error: code = NotFound desc = could not find container \"3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87\": container with ID starting with 3e8871ee9865a8c4a164e723d9617e7e53500c1da7dc55b3b2b7719466132b87 not found: ID does not exist"
Oct 30 18:47:26.207163 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.207139    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:47:26.207265 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.207169    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9tmq\" (UniqueName: \"kubernetes.io/projected/101ee503-0f76-4a6b-af62-35f62eff638f-kube-api-access-v9tmq\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.215215 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.215189    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"]
Oct 30 18:47:26.220527 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.220502    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5c767fcc7f-9w8pn"]
Oct 30 18:47:26.222751 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.222734    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:47:26.265255 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.265200    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/101ee503-0f76-4a6b-af62-35f62eff638f-plugins" pod "101ee503-0f76-4a6b-af62-35f62eff638f" (UID: "101ee503-0f76-4a6b-af62-35f62eff638f"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:47:26.275972 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.275901    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"]
Oct 30 18:47:26.279924 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.279890    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-rpbs7"]
Oct 30 18:47:26.308311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.308281    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:47:26.308311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.308307    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/101ee503-0f76-4a6b-af62-35f62eff638f-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:26.328621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.328591    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:26.335263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.335238    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-collector-1"]
Oct 30 18:47:26.517958 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.513494    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:47:26.524043 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:26.524015    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:47:28.007512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:28.007475    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07232107-7667-4721-abe8-6b5466d95f6c" path="/var/lib/kubelet/pods/07232107-7667-4721-abe8-6b5466d95f6c/volumes"
Oct 30 18:47:28.007921 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:28.007871    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="101ee503-0f76-4a6b-af62-35f62eff638f" path="/var/lib/kubelet/pods/101ee503-0f76-4a6b-af62-35f62eff638f/volumes"
Oct 30 18:47:28.008159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:28.008146    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0b04cd0-b2e9-4335-ab23-2acc4bd29999" path="/var/lib/kubelet/pods/a0b04cd0-b2e9-4335-ab23-2acc4bd29999/volumes"
Oct 30 18:47:28.008385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:28.008374    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fde1e29e-0b9f-4713-94fa-c6edc979ccb7" path="/var/lib/kubelet/pods/fde1e29e-0b9f-4713-94fa-c6edc979ccb7/volumes"
Oct 30 18:47:35.141738 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.141706    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"]
Oct 30 18:47:35.142229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.142017    5746 cpu_manager.go:414] "containerMap: removing container" podUID="101ee503-0f76-4a6b-af62-35f62eff638f" containerName="perses"
Oct 30 18:47:35.142229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.142031    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="101ee503-0f76-4a6b-af62-35f62eff638f" containerName="perses"
Oct 30 18:47:35.142229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.142045    5746 cpu_manager.go:414] "containerMap: removing container" podUID="fde1e29e-0b9f-4713-94fa-c6edc979ccb7" containerName="ta-container"
Oct 30 18:47:35.142229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.142050    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fde1e29e-0b9f-4713-94fa-c6edc979ccb7" containerName="ta-container"
Oct 30 18:47:35.142229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.142057    5746 cpu_manager.go:414] "containerMap: removing container" podUID="07232107-7667-4721-abe8-6b5466d95f6c" containerName="otc-container"
Oct 30 18:47:35.142229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.142062    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07232107-7667-4721-abe8-6b5466d95f6c" containerName="otc-container"
Oct 30 18:47:35.146927 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.143045    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="101ee503-0f76-4a6b-af62-35f62eff638f" containerName="perses"
Oct 30 18:47:35.146927 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.143123    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="fde1e29e-0b9f-4713-94fa-c6edc979ccb7" containerName="ta-container"
Oct 30 18:47:35.146927 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.143149    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="07232107-7667-4721-abe8-6b5466d95f6c" containerName="otc-container"
Oct 30 18:47:35.148291 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.147443    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"
Oct 30 18:47:35.163040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.163015    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"]
Oct 30 18:47:35.174591 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.174567    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:47:35.174591 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.174574    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:47:35.174733 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.174630    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:47:35.282816 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.282780    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhscw\" (UniqueName: \"kubernetes.io/projected/3644c7bb-d9d2-49bd-8615-81c6a59fe7a8-kube-api-access-jhscw\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj\" (UID: \"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"
Oct 30 18:47:35.383428 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.383392    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhscw\" (UniqueName: \"kubernetes.io/projected/3644c7bb-d9d2-49bd-8615-81c6a59fe7a8-kube-api-access-jhscw\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj\" (UID: \"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"
Oct 30 18:47:35.411955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.411871    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhscw\" (UniqueName: \"kubernetes.io/projected/3644c7bb-d9d2-49bd-8615-81c6a59fe7a8-kube-api-access-jhscw\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj\" (UID: \"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"
Oct 30 18:47:35.459835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.459807    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"
Oct 30 18:47:35.752031 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:35.751961    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"]
Oct 30 18:47:35.755762 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:47:35.755731    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3644c7bb_d9d2_49bd_8615_81c6a59fe7a8.slice/crio-eb6d31009074ce8e6f23b57c7766b60846e7bac9c3f3b368bb77882ce372bf9f WatchSource:0}: Error finding container eb6d31009074ce8e6f23b57c7766b60846e7bac9c3f3b368bb77882ce372bf9f: Status 404 returned error can't find the container with id eb6d31009074ce8e6f23b57c7766b60846e7bac9c3f3b368bb77882ce372bf9f
Oct 30 18:47:36.089447 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.089373    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:47:36.092567 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.092545    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.106869 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.106844    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:47:36.112535 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.111996    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:47:36.183170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.183136    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"]
Oct 30 18:47:36.186317 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.186298    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.188796 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.188778    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-plugins\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.188925 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.188806    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-config\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.188925 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.188832    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5h5w\" (UniqueName: \"kubernetes.io/projected/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-kube-api-access-h5h5w\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.201022 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.201004    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:47:36.206152 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.206129    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"]
Oct 30 18:47:36.212634 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.212608    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj" event={"ID":"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8","Type":"ContainerStarted","Data":"884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a"}
Oct 30 18:47:36.212761 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.212647    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj" event={"ID":"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8","Type":"ContainerStarted","Data":"eb6d31009074ce8e6f23b57c7766b60846e7bac9c3f3b368bb77882ce372bf9f"}
Oct 30 18:47:36.227816 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.227787    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-tk8r4\""
Oct 30 18:47:36.227816 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.227790    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:47:36.264295 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.264246    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj" podStartSLOduration=1.264232891 podStartE2EDuration="1.264232891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:36.263619014 +0000 UTC m=+3026.430437142" watchObservedRunningTime="2026-10-30 18:47:36.264232891 +0000 UTC m=+3026.431051017"
Oct 30 18:47:36.290011 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.289983    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-plugins\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.290177 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.290034    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-config\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.290218 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.290182    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5h5w\" (UniqueName: \"kubernetes.io/projected/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-kube-api-access-h5h5w\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.290277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.290233    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmt44\" (UniqueName: \"kubernetes.io/projected/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-kube-api-access-vmt44\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.290418 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.290382    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.290499 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.290414    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-plugins\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.290572 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.290506    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-ta-internal\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.290731 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.290714    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-config\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.314800 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.314770    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5h5w\" (UniqueName: \"kubernetes.io/projected/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-kube-api-access-h5h5w\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:36.390901 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.390865    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmt44\" (UniqueName: \"kubernetes.io/projected/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-kube-api-access-vmt44\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.391137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.390963    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.391137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.390991    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-ta-internal\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.391624 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.391600    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-ta-internal\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.393536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.393514    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.413756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.413728    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmt44\" (UniqueName: \"kubernetes.io/projected/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-kube-api-access-vmt44\") pod \"data-science-collector-targetallocator-9d866c9f4-lzcx5\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.498074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.498035    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:36.794003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.793893    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"]
Oct 30 18:47:36.797584 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:47:36.797555    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09f4ac79_4b2c_4a88_a7bd_b80ed86cb4e0.slice/crio-c4617559b6e7e3afa78db286a60358cc4e0ead7b5172aa762debb171810fae59 WatchSource:0}: Error finding container c4617559b6e7e3afa78db286a60358cc4e0ead7b5172aa762debb171810fae59: Status 404 returned error can't find the container with id c4617559b6e7e3afa78db286a60358cc4e0ead7b5172aa762debb171810fae59
Oct 30 18:47:36.799948 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.799878    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:47:36.800044 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:36.799993    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:47:37.218683 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:37.218646    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5" event={"ID":"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0","Type":"ContainerStarted","Data":"82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf"}
Oct 30 18:47:37.219138 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:37.218697    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5" event={"ID":"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0","Type":"ContainerStarted","Data":"c4617559b6e7e3afa78db286a60358cc4e0ead7b5172aa762debb171810fae59"}
Oct 30 18:47:37.219138 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:37.218810    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:37.220430 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:37.220410    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:47:37.259532 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:37.259469    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5" podStartSLOduration=1.259452892 podStartE2EDuration="1.259452892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:37.258670357 +0000 UTC m=+3027.425488495" watchObservedRunningTime="2026-10-30 18:47:37.259452892 +0000 UTC m=+3027.426271020"
Oct 30 18:47:42.789689 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:42.789653    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:47:42.938811 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:42.938778    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"]
Oct 30 18:47:42.942072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:42.942051    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"
Oct 30 18:47:42.959898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:42.959874    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"]
Oct 30 18:47:43.047522 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.047449    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl72b\" (UniqueName: \"kubernetes.io/projected/38c544ec-f256-40ea-a177-c548c7e716c2-kube-api-access-nl72b\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt\" (UID: \"38c544ec-f256-40ea-a177-c548c7e716c2\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"
Oct 30 18:47:43.148120 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.148081    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:43.148290 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.148145    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl72b\" (UniqueName: \"kubernetes.io/projected/38c544ec-f256-40ea-a177-c548c7e716c2-kube-api-access-nl72b\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt\" (UID: \"38c544ec-f256-40ea-a177-c548c7e716c2\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"
Oct 30 18:47:43.171364 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.171331    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl72b\" (UniqueName: \"kubernetes.io/projected/38c544ec-f256-40ea-a177-c548c7e716c2-kube-api-access-nl72b\") pod \"thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt\" (UID: \"38c544ec-f256-40ea-a177-c548c7e716c2\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"
Oct 30 18:47:43.176042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.176024    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:43.248898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.248873    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:43.249031 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.249013    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:43.253857 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.253832    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"
Oct 30 18:47:43.264600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.264579    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:43.371961 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.371936    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:43.402823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.402797    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:43.549313 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.549153    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"]
Oct 30 18:47:43.551900 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:47:43.551835    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38c544ec_f256_40ea_a177_c548c7e716c2.slice/crio-4f230a29bb353ef22c615d7ddd1669fcff8283b039d69b32f833c3c4ac1ce30b WatchSource:0}: Error finding container 4f230a29bb353ef22c615d7ddd1669fcff8283b039d69b32f833c3c4ac1ce30b: Status 404 returned error can't find the container with id 4f230a29bb353ef22c615d7ddd1669fcff8283b039d69b32f833c3c4ac1ce30b
Oct 30 18:47:43.604278 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.604258    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:43.902544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:43.902516    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:47:43.905733 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:47:43.905700    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68d4b3dd_7845_4ad7_bb11_42caad5c5da9.slice/crio-470e965457111b1e7fd01ef949c7f4bdf7e7e1815e6e2278a34ad6e287d20d84 WatchSource:0}: Error finding container 470e965457111b1e7fd01ef949c7f4bdf7e7e1815e6e2278a34ad6e287d20d84: Status 404 returned error can't find the container with id 470e965457111b1e7fd01ef949c7f4bdf7e7e1815e6e2278a34ad6e287d20d84
Oct 30 18:47:44.248547 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.248462    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"68d4b3dd-7845-4ad7-bb11-42caad5c5da9","Type":"ContainerStarted","Data":"bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c"}
Oct 30 18:47:44.248547 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.248520    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:47:44.248547 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.248540    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"68d4b3dd-7845-4ad7-bb11-42caad5c5da9","Type":"ContainerStarted","Data":"470e965457111b1e7fd01ef949c7f4bdf7e7e1815e6e2278a34ad6e287d20d84"}
Oct 30 18:47:44.251099 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.251071    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt" event={"ID":"38c544ec-f256-40ea-a177-c548c7e716c2","Type":"ContainerStarted","Data":"04b773ffb6f9c470be15b8a1a990caea5f30a3074788538c9514389582733428"}
Oct 30 18:47:44.251243 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.251111    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt" event={"ID":"38c544ec-f256-40ea-a177-c548c7e716c2","Type":"ContainerStarted","Data":"4f230a29bb353ef22c615d7ddd1669fcff8283b039d69b32f833c3c4ac1ce30b"}
Oct 30 18:47:44.288834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.288777    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=8.288761104 podStartE2EDuration="8.288761104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:44.288323824 +0000 UTC m=+3034.455141970" watchObservedRunningTime="2026-10-30 18:47:44.288761104 +0000 UTC m=+3034.455579230"
Oct 30 18:47:44.326780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.326727    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt" podStartSLOduration=2.326712593 podStartE2EDuration="2.326712593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:44.326522962 +0000 UTC m=+3034.493341089" watchObservedRunningTime="2026-10-30 18:47:44.326712593 +0000 UTC m=+3034.493530720"
Oct 30 18:47:44.352337 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.352262    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"]
Oct 30 18:47:44.352479 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.352447    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj" podUID="3644c7bb-d9d2-49bd-8615-81c6a59fe7a8" containerName="thanos-querier" containerID="cri-o://884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a" gracePeriod=30
Oct 30 18:47:44.613018 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.612991    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"
Oct 30 18:47:44.663219 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.663184    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3644c7bb-d9d2-49bd-8615-81c6a59fe7a8-kube-api-access-jhscw\" (UniqueName: \"kubernetes.io/projected/3644c7bb-d9d2-49bd-8615-81c6a59fe7a8-kube-api-access-jhscw\") pod \"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8\" (UID: \"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8\") "
Oct 30 18:47:44.665407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.665371    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3644c7bb-d9d2-49bd-8615-81c6a59fe7a8-kube-api-access-jhscw" pod "3644c7bb-d9d2-49bd-8615-81c6a59fe7a8" (UID: "3644c7bb-d9d2-49bd-8615-81c6a59fe7a8"). InnerVolumeSpecName "kube-api-access-jhscw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:44.764750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:44.764715    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhscw\" (UniqueName: \"kubernetes.io/projected/3644c7bb-d9d2-49bd-8615-81c6a59fe7a8-kube-api-access-jhscw\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:47:45.257397 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.257364    5746 generic.go:356] "Generic (PLEG): container finished" podID="3644c7bb-d9d2-49bd-8615-81c6a59fe7a8" containerID="884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a" exitCode=0
Oct 30 18:47:45.257901 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.257414    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"
Oct 30 18:47:45.257901 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.257453    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj" event={"ID":"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8","Type":"ContainerDied","Data":"884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a"}
Oct 30 18:47:45.257901 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.257505    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj" event={"ID":"3644c7bb-d9d2-49bd-8615-81c6a59fe7a8","Type":"ContainerDied","Data":"eb6d31009074ce8e6f23b57c7766b60846e7bac9c3f3b368bb77882ce372bf9f"}
Oct 30 18:47:45.257901 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.257523    5746 scope.go:122] "RemoveContainer" containerID="884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a"
Oct 30 18:47:45.267358 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.267342    5746 scope.go:122] "RemoveContainer" containerID="884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a"
Oct 30 18:47:45.267638 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:47:45.267617    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a\": container with ID starting with 884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a not found: ID does not exist" containerID="884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a"
Oct 30 18:47:45.267709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.267649    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a"} err="failed to get container status \"884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a\": rpc error: code = NotFound desc = could not find container \"884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a\": container with ID starting with 884845574ed104c431b8be707a5878e2d39d8bf89853c43ccbd047f85333702a not found: ID does not exist"
Oct 30 18:47:45.306707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.306679    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"]
Oct 30 18:47:45.318467 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:45.318439    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-rbptj"]
Oct 30 18:47:46.010553 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:46.010508    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3644c7bb-d9d2-49bd-8615-81c6a59fe7a8" path="/var/lib/kubelet/pods/3644c7bb-d9d2-49bd-8615-81c6a59fe7a8/volumes"
Oct 30 18:47:55.259795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:47:55.259765    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:48:05.073207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.073170    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"]
Oct 30 18:48:05.073713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.073471    5746 cpu_manager.go:414] "containerMap: removing container" podUID="3644c7bb-d9d2-49bd-8615-81c6a59fe7a8" containerName="thanos-querier"
Oct 30 18:48:05.073713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.073484    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3644c7bb-d9d2-49bd-8615-81c6a59fe7a8" containerName="thanos-querier"
Oct 30 18:48:05.073713 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.073577    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="3644c7bb-d9d2-49bd-8615-81c6a59fe7a8" containerName="thanos-querier"
Oct 30 18:48:05.093419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.093393    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.099112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.099089    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"]
Oct 30 18:48:05.235097 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.235062    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4pcm\" (UniqueName: \"kubernetes.io/projected/7e0c0381-397e-4d74-876b-932d0d8a126d-kube-api-access-z4pcm\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.235097 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.235109    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/7e0c0381-397e-4d74-876b-932d0d8a126d-ta-internal\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.235346 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.235140    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7e0c0381-397e-4d74-876b-932d0d8a126d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.336634 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.336558    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7e0c0381-397e-4d74-876b-932d0d8a126d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.336799 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.336635    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4pcm\" (UniqueName: \"kubernetes.io/projected/7e0c0381-397e-4d74-876b-932d0d8a126d-kube-api-access-z4pcm\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.336799 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.336658    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/7e0c0381-397e-4d74-876b-932d0d8a126d-ta-internal\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.337263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.337243    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/7e0c0381-397e-4d74-876b-932d0d8a126d-ta-internal\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.339195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.339173    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7e0c0381-397e-4d74-876b-932d0d8a126d-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.359951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.359922    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4pcm\" (UniqueName: \"kubernetes.io/projected/7e0c0381-397e-4d74-876b-932d0d8a126d-kube-api-access-z4pcm\") pod \"data-science-collector-targetallocator-7b4cbb66c8-gxm95\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.405809 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.405788    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:05.702466 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.702364    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"]
Oct 30 18:48:05.705122 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:48:05.705092    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e0c0381_397e_4d74_876b_932d0d8a126d.slice/crio-bab1b28e8648c442c80d2456a80df8f40285b6a61d4e192d806a18d3d917befb WatchSource:0}: Error finding container bab1b28e8648c442c80d2456a80df8f40285b6a61d4e192d806a18d3d917befb: Status 404 returned error can't find the container with id bab1b28e8648c442c80d2456a80df8f40285b6a61d4e192d806a18d3d917befb
Oct 30 18:48:05.707169 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.707135    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:48:05.707269 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:05.707205    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:48:06.349440 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.349410    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95" event={"ID":"7e0c0381-397e-4d74-876b-932d0d8a126d","Type":"ContainerStarted","Data":"44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db"}
Oct 30 18:48:06.349828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.349454    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95" event={"ID":"7e0c0381-397e-4d74-876b-932d0d8a126d","Type":"ContainerStarted","Data":"bab1b28e8648c442c80d2456a80df8f40285b6a61d4e192d806a18d3d917befb"}
Oct 30 18:48:06.349828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.349585    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:06.351274 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.351253    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:06.392429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.392375    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95" podStartSLOduration=1.392358298 podStartE2EDuration="1.392358298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:06.391670565 +0000 UTC m=+3056.558488691" watchObservedRunningTime="2026-10-30 18:48:06.392358298 +0000 UTC m=+3056.559176429"
Oct 30 18:48:06.482371 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.482335    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"]
Oct 30 18:48:06.482557 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.482513    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5" podUID="09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" containerName="ta-container" containerID="cri-o://82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf" gracePeriod=30
Oct 30 18:48:06.726116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.726088    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:48:06.849804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.849772    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-kube-api-access-vmt44\" (UniqueName: \"kubernetes.io/projected/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-kube-api-access-vmt44\") pod \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") "
Oct 30 18:48:06.850027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.849827    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-ta-internal\" (UniqueName: \"kubernetes.io/configmap/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-ta-internal\") pod \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") "
Oct 30 18:48:06.850027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.849873    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-data-science-collector-ta-server-cert\") pod \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\" (UID: \"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0\") "
Oct 30 18:48:06.850231 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.850208    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-ta-internal" pod "09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" (UID: "09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:06.852193 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.852133    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-kube-api-access-vmt44" pod "09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" (UID: "09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0"). InnerVolumeSpecName "kube-api-access-vmt44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:06.852289 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.852237    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-data-science-collector-ta-server-cert" pod "09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" (UID: "09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:06.950580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.950553    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:06.950580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.950577    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmt44\" (UniqueName: \"kubernetes.io/projected/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-kube-api-access-vmt44\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:06.950756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:06.950587    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:07.355017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.354980    5746 generic.go:356] "Generic (PLEG): container finished" podID="09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" containerID="82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf" exitCode=0
Oct 30 18:48:07.355446 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.355038    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"
Oct 30 18:48:07.355446 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.355062    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5" event={"ID":"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0","Type":"ContainerDied","Data":"82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf"}
Oct 30 18:48:07.355446 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.355108    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5" event={"ID":"09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0","Type":"ContainerDied","Data":"c4617559b6e7e3afa78db286a60358cc4e0ead7b5172aa762debb171810fae59"}
Oct 30 18:48:07.355446 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.355127    5746 scope.go:122] "RemoveContainer" containerID="82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf"
Oct 30 18:48:07.363862 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.363839    5746 scope.go:122] "RemoveContainer" containerID="82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf"
Oct 30 18:48:07.364173 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:48:07.364155    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf\": container with ID starting with 82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf not found: ID does not exist" containerID="82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf"
Oct 30 18:48:07.364246 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.364186    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf"} err="failed to get container status \"82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf\": rpc error: code = NotFound desc = could not find container \"82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf\": container with ID starting with 82d9b2b3886af21e508502a33b80db8179c8dca058a3dec7d0944dd315830bdf not found: ID does not exist"
Oct 30 18:48:07.404510 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.404479    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"]
Oct 30 18:48:07.416369 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:07.416345    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-lzcx5"]
Oct 30 18:48:08.008002 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:08.007971    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" path="/var/lib/kubelet/pods/09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0/volumes"
Oct 30 18:48:12.420050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.420017    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"]
Oct 30 18:48:12.420473 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.420327    5746 cpu_manager.go:414] "containerMap: removing container" podUID="09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" containerName="ta-container"
Oct 30 18:48:12.420473 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.420340    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" containerName="ta-container"
Oct 30 18:48:12.420473 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.420406    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="09f4ac79-4b2c-4a88-a7bd-b80ed86cb4e0" containerName="ta-container"
Oct 30 18:48:12.426155 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.426134    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.454947 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.454901    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"]
Oct 30 18:48:12.488235 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.488207    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/d90677fd-8623-42a7-a797-2e2e6d850f2a-ta-internal\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.488235 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.488239    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d90677fd-8623-42a7-a797-2e2e6d850f2a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.488414 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.488308    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8n7n\" (UniqueName: \"kubernetes.io/projected/d90677fd-8623-42a7-a797-2e2e6d850f2a-kube-api-access-x8n7n\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.589437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.589402    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/d90677fd-8623-42a7-a797-2e2e6d850f2a-ta-internal\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.589437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.589436    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d90677fd-8623-42a7-a797-2e2e6d850f2a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.589692 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.589454    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8n7n\" (UniqueName: \"kubernetes.io/projected/d90677fd-8623-42a7-a797-2e2e6d850f2a-kube-api-access-x8n7n\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.590142 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.590118    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/d90677fd-8623-42a7-a797-2e2e6d850f2a-ta-internal\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.592127 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.592107    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d90677fd-8623-42a7-a797-2e2e6d850f2a-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.612745 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.612714    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8n7n\" (UniqueName: \"kubernetes.io/projected/d90677fd-8623-42a7-a797-2e2e6d850f2a-kube-api-access-x8n7n\") pod \"data-science-collector-targetallocator-9d866c9f4-8nq9s\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.737180 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:12.737098    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:12.802944 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:48:12.801096    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:48:13.031469 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:13.031384    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"]
Oct 30 18:48:13.035114 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:48:13.035085    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd90677fd_8623_42a7_a797_2e2e6d850f2a.slice/crio-2425ec3502616b0a500a9b7534b61ea9b7c2d380925c50874f9a92e0aaadc66e WatchSource:0}: Error finding container 2425ec3502616b0a500a9b7534b61ea9b7c2d380925c50874f9a92e0aaadc66e: Status 404 returned error can't find the container with id 2425ec3502616b0a500a9b7534b61ea9b7c2d380925c50874f9a92e0aaadc66e
Oct 30 18:48:13.037499 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:13.037468    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:48:13.037586 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:13.037533    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:48:13.384410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:13.384379    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s" event={"ID":"d90677fd-8623-42a7-a797-2e2e6d850f2a","Type":"ContainerStarted","Data":"2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26"}
Oct 30 18:48:13.384606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:13.384423    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s" event={"ID":"d90677fd-8623-42a7-a797-2e2e6d850f2a","Type":"ContainerStarted","Data":"2425ec3502616b0a500a9b7534b61ea9b7c2d380925c50874f9a92e0aaadc66e"}
Oct 30 18:48:13.384606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:13.384576    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:13.385712 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:13.385680    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s" podUID="d90677fd-8623-42a7-a797-2e2e6d850f2a" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.140:8080/readyz\": dial tcp 10.133.0.140:8080: connect: connection refused"
Oct 30 18:48:13.424601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:13.424551    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s" podStartSLOduration=1.424539451 podStartE2EDuration="1.424539451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:13.424376947 +0000 UTC m=+3063.591195098" watchObservedRunningTime="2026-10-30 18:48:13.424539451 +0000 UTC m=+3063.591357579"
Oct 30 18:48:14.389679 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.389652    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:14.461901 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.461870    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"]
Oct 30 18:48:14.462321 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.462059    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95" podUID="7e0c0381-397e-4d74-876b-932d0d8a126d" containerName="ta-container" containerID="cri-o://44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db" gracePeriod=30
Oct 30 18:48:14.699356 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.699333    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:14.704369 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.704347    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e0c0381-397e-4d74-876b-932d0d8a126d-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7e0c0381-397e-4d74-876b-932d0d8a126d-data-science-collector-ta-server-cert\") pod \"7e0c0381-397e-4d74-876b-932d0d8a126d\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") "
Oct 30 18:48:14.704442 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.704420    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e0c0381-397e-4d74-876b-932d0d8a126d-kube-api-access-z4pcm\" (UniqueName: \"kubernetes.io/projected/7e0c0381-397e-4d74-876b-932d0d8a126d-kube-api-access-z4pcm\") pod \"7e0c0381-397e-4d74-876b-932d0d8a126d\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") "
Oct 30 18:48:14.706419 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.706393    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e0c0381-397e-4d74-876b-932d0d8a126d-data-science-collector-ta-server-cert" pod "7e0c0381-397e-4d74-876b-932d0d8a126d" (UID: "7e0c0381-397e-4d74-876b-932d0d8a126d"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:14.706509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.706493    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e0c0381-397e-4d74-876b-932d0d8a126d-kube-api-access-z4pcm" pod "7e0c0381-397e-4d74-876b-932d0d8a126d" (UID: "7e0c0381-397e-4d74-876b-932d0d8a126d"). InnerVolumeSpecName "kube-api-access-z4pcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:14.805015 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.804984    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e0c0381-397e-4d74-876b-932d0d8a126d-ta-internal\" (UniqueName: \"kubernetes.io/configmap/7e0c0381-397e-4d74-876b-932d0d8a126d-ta-internal\") pod \"7e0c0381-397e-4d74-876b-932d0d8a126d\" (UID: \"7e0c0381-397e-4d74-876b-932d0d8a126d\") "
Oct 30 18:48:14.805183 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.805129    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4pcm\" (UniqueName: \"kubernetes.io/projected/7e0c0381-397e-4d74-876b-932d0d8a126d-kube-api-access-z4pcm\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:14.805183 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.805141    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/7e0c0381-397e-4d74-876b-932d0d8a126d-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:14.805370 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.805346    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e0c0381-397e-4d74-876b-932d0d8a126d-ta-internal" pod "7e0c0381-397e-4d74-876b-932d0d8a126d" (UID: "7e0c0381-397e-4d74-876b-932d0d8a126d"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:14.905671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:14.905633    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/7e0c0381-397e-4d74-876b-932d0d8a126d-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:15.393887 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.393853    5746 generic.go:356] "Generic (PLEG): container finished" podID="7e0c0381-397e-4d74-876b-932d0d8a126d" containerID="44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db" exitCode=0
Oct 30 18:48:15.394085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.393929    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"
Oct 30 18:48:15.394085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.393946    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95" event={"ID":"7e0c0381-397e-4d74-876b-932d0d8a126d","Type":"ContainerDied","Data":"44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db"}
Oct 30 18:48:15.394085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.393987    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95" event={"ID":"7e0c0381-397e-4d74-876b-932d0d8a126d","Type":"ContainerDied","Data":"bab1b28e8648c442c80d2456a80df8f40285b6a61d4e192d806a18d3d917befb"}
Oct 30 18:48:15.394085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.394004    5746 scope.go:122] "RemoveContainer" containerID="44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db"
Oct 30 18:48:15.402598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.402581    5746 scope.go:122] "RemoveContainer" containerID="44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db"
Oct 30 18:48:15.402881 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:48:15.402859    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db\": container with ID starting with 44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db not found: ID does not exist" containerID="44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db"
Oct 30 18:48:15.402998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.402890    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db"} err="failed to get container status \"44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db\": rpc error: code = NotFound desc = could not find container \"44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db\": container with ID starting with 44e8050a6a7f3282c9572b671db23236e9cf7e1824fdddce80a764bf51adb6db not found: ID does not exist"
Oct 30 18:48:15.443782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.443751    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"]
Oct 30 18:48:15.455371 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:15.455351    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7b4cbb66c8-gxm95"]
Oct 30 18:48:16.008151 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:16.008113    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e0c0381-397e-4d74-876b-932d0d8a126d" path="/var/lib/kubelet/pods/7e0c0381-397e-4d74-876b-932d0d8a126d/volumes"
Oct 30 18:48:42.789761 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:48:42.789720    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:48:54.374291 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:54.374257    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"]
Oct 30 18:48:54.374745 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:54.374483    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt" podUID="38c544ec-f256-40ea-a177-c548c7e716c2" containerName="thanos-querier" containerID="cri-o://04b773ffb6f9c470be15b8a1a990caea5f30a3074788538c9514389582733428" gracePeriod=30
Oct 30 18:48:54.551768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:54.551744    5746 generic.go:356] "Generic (PLEG): container finished" podID="38c544ec-f256-40ea-a177-c548c7e716c2" containerID="04b773ffb6f9c470be15b8a1a990caea5f30a3074788538c9514389582733428" exitCode=0
Oct 30 18:48:54.551874 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:54.551812    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt" event={"ID":"38c544ec-f256-40ea-a177-c548c7e716c2","Type":"ContainerDied","Data":"04b773ffb6f9c470be15b8a1a990caea5f30a3074788538c9514389582733428"}
Oct 30 18:48:54.631413 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:54.631357    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"
Oct 30 18:48:54.721528 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:54.721372    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38c544ec-f256-40ea-a177-c548c7e716c2-kube-api-access-nl72b\" (UniqueName: \"kubernetes.io/projected/38c544ec-f256-40ea-a177-c548c7e716c2-kube-api-access-nl72b\") pod \"38c544ec-f256-40ea-a177-c548c7e716c2\" (UID: \"38c544ec-f256-40ea-a177-c548c7e716c2\") "
Oct 30 18:48:54.723776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:54.723752    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c544ec-f256-40ea-a177-c548c7e716c2-kube-api-access-nl72b" pod "38c544ec-f256-40ea-a177-c548c7e716c2" (UID: "38c544ec-f256-40ea-a177-c548c7e716c2"). InnerVolumeSpecName "kube-api-access-nl72b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:54.822137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:54.822104    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl72b\" (UniqueName: \"kubernetes.io/projected/38c544ec-f256-40ea-a177-c548c7e716c2-kube-api-access-nl72b\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:55.142701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.142637    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"]
Oct 30 18:48:55.142972 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.142943    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s" podUID="d90677fd-8623-42a7-a797-2e2e6d850f2a" containerName="ta-container" containerID="cri-o://2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26" gracePeriod=30
Oct 30 18:48:55.231432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.230165    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:48:55.237719 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.237212    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="68d4b3dd-7845-4ad7-bb11-42caad5c5da9" containerName="perses" containerID="cri-o://bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c" gracePeriod=30
Oct 30 18:48:55.259345 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.259315    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-perses-0" podUID="68d4b3dd-7845-4ad7-bb11-42caad5c5da9" containerName="perses" probeResult="failure" output="Get \"http://10.133.0.136:8080/metrics\": dial tcp 10.133.0.136:8080: connect: connection refused"
Oct 30 18:48:55.404110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.404046    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:55.503352 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.503321    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:48:55.527375 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.527346    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d90677fd-8623-42a7-a797-2e2e6d850f2a-kube-api-access-x8n7n\" (UniqueName: \"kubernetes.io/projected/d90677fd-8623-42a7-a797-2e2e6d850f2a-kube-api-access-x8n7n\") pod \"d90677fd-8623-42a7-a797-2e2e6d850f2a\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") "
Oct 30 18:48:55.527546 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.527384    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d90677fd-8623-42a7-a797-2e2e6d850f2a-ta-internal\" (UniqueName: \"kubernetes.io/configmap/d90677fd-8623-42a7-a797-2e2e6d850f2a-ta-internal\") pod \"d90677fd-8623-42a7-a797-2e2e6d850f2a\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") "
Oct 30 18:48:55.527546 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.527406    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d90677fd-8623-42a7-a797-2e2e6d850f2a-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d90677fd-8623-42a7-a797-2e2e6d850f2a-data-science-collector-ta-server-cert\") pod \"d90677fd-8623-42a7-a797-2e2e6d850f2a\" (UID: \"d90677fd-8623-42a7-a797-2e2e6d850f2a\") "
Oct 30 18:48:55.527827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.527792    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d90677fd-8623-42a7-a797-2e2e6d850f2a-ta-internal" pod "d90677fd-8623-42a7-a797-2e2e6d850f2a" (UID: "d90677fd-8623-42a7-a797-2e2e6d850f2a"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:55.530160 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.530134    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d90677fd-8623-42a7-a797-2e2e6d850f2a-kube-api-access-x8n7n" pod "d90677fd-8623-42a7-a797-2e2e6d850f2a" (UID: "d90677fd-8623-42a7-a797-2e2e6d850f2a"). InnerVolumeSpecName "kube-api-access-x8n7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:55.530244 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.530134    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d90677fd-8623-42a7-a797-2e2e6d850f2a-data-science-collector-ta-server-cert" pod "d90677fd-8623-42a7-a797-2e2e6d850f2a" (UID: "d90677fd-8623-42a7-a797-2e2e6d850f2a"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:55.557948 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.557894    5746 generic.go:356] "Generic (PLEG): container finished" podID="68d4b3dd-7845-4ad7-bb11-42caad5c5da9" containerID="bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c" exitCode=0
Oct 30 18:48:55.558124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.557951    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"68d4b3dd-7845-4ad7-bb11-42caad5c5da9","Type":"ContainerDied","Data":"bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c"}
Oct 30 18:48:55.558124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.557982    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:48:55.558124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.558001    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"68d4b3dd-7845-4ad7-bb11-42caad5c5da9","Type":"ContainerDied","Data":"470e965457111b1e7fd01ef949c7f4bdf7e7e1815e6e2278a34ad6e287d20d84"}
Oct 30 18:48:55.558124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.558024    5746 scope.go:122] "RemoveContainer" containerID="bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c"
Oct 30 18:48:55.559412 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.559390    5746 generic.go:356] "Generic (PLEG): container finished" podID="d90677fd-8623-42a7-a797-2e2e6d850f2a" containerID="2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26" exitCode=0
Oct 30 18:48:55.559536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.559434    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"
Oct 30 18:48:55.559536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.559478    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s" event={"ID":"d90677fd-8623-42a7-a797-2e2e6d850f2a","Type":"ContainerDied","Data":"2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26"}
Oct 30 18:48:55.559536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.559517    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s" event={"ID":"d90677fd-8623-42a7-a797-2e2e6d850f2a","Type":"ContainerDied","Data":"2425ec3502616b0a500a9b7534b61ea9b7c2d380925c50874f9a92e0aaadc66e"}
Oct 30 18:48:55.560984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.560958    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt" event={"ID":"38c544ec-f256-40ea-a177-c548c7e716c2","Type":"ContainerDied","Data":"4f230a29bb353ef22c615d7ddd1669fcff8283b039d69b32f833c3c4ac1ce30b"}
Oct 30 18:48:55.561118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.560965    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"
Oct 30 18:48:55.566754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.566737    5746 scope.go:122] "RemoveContainer" containerID="bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c"
Oct 30 18:48:55.567067 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:48:55.567046    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c\": container with ID starting with bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c not found: ID does not exist" containerID="bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c"
Oct 30 18:48:55.567160 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.567075    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c"} err="failed to get container status \"bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c\": rpc error: code = NotFound desc = could not find container \"bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c\": container with ID starting with bea3db81d85715f8bde9ace56f94e4af1e30bdc2ab33f84a737973e537add01c not found: ID does not exist"
Oct 30 18:48:55.567160 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.567090    5746 scope.go:122] "RemoveContainer" containerID="2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26"
Oct 30 18:48:55.575142 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.575124    5746 scope.go:122] "RemoveContainer" containerID="2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26"
Oct 30 18:48:55.575387 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:48:55.575367    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26\": container with ID starting with 2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26 not found: ID does not exist" containerID="2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26"
Oct 30 18:48:55.575459 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.575393    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26"} err="failed to get container status \"2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26\": rpc error: code = NotFound desc = could not find container \"2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26\": container with ID starting with 2de4fe1f515b74606de7cdd172e496417def289c5bbc2d1a56f63e736267ce26 not found: ID does not exist"
Oct 30 18:48:55.575459 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.575409    5746 scope.go:122] "RemoveContainer" containerID="04b773ffb6f9c470be15b8a1a990caea5f30a3074788538c9514389582733428"
Oct 30 18:48:55.621805 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.621778    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"]
Oct 30 18:48:55.626707 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.626683    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-9d866c9f4-8nq9s"]
Oct 30 18:48:55.628083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.628061    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-plugins\" (UniqueName: \"kubernetes.io/empty-dir/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-plugins\") pod \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") "
Oct 30 18:48:55.628171 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.628114    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-config\" (UniqueName: \"kubernetes.io/configmap/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-config\") pod \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") "
Oct 30 18:48:55.628171 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.628150    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-kube-api-access-h5h5w\" (UniqueName: \"kubernetes.io/projected/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-kube-api-access-h5h5w\") pod \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") "
Oct 30 18:48:55.628261 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.628224    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\" (UID: \"68d4b3dd-7845-4ad7-bb11-42caad5c5da9\") "
Oct 30 18:48:55.628440 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.628418    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8n7n\" (UniqueName: \"kubernetes.io/projected/d90677fd-8623-42a7-a797-2e2e6d850f2a-kube-api-access-x8n7n\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:55.628524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.628440    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/d90677fd-8623-42a7-a797-2e2e6d850f2a-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:55.628524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.628455    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/d90677fd-8623-42a7-a797-2e2e6d850f2a-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:55.628524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.628481    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-config" pod "68d4b3dd-7845-4ad7-bb11-42caad5c5da9" (UID: "68d4b3dd-7845-4ad7-bb11-42caad5c5da9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:55.630545 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.630523    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-kube-api-access-h5h5w" pod "68d4b3dd-7845-4ad7-bb11-42caad5c5da9" (UID: "68d4b3dd-7845-4ad7-bb11-42caad5c5da9"). InnerVolumeSpecName "kube-api-access-h5h5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:55.631276 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.631257    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "68d4b3dd-7845-4ad7-bb11-42caad5c5da9" (UID: "68d4b3dd-7845-4ad7-bb11-42caad5c5da9"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:48:55.658803 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.658751    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"]
Oct 30 18:48:55.663221 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.663199    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-74b6964bcd-n2hwt"]
Oct 30 18:48:55.729004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.728976    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:55.729004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.728997    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5h5w\" (UniqueName: \"kubernetes.io/projected/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-kube-api-access-h5h5w\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:55.729175 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.729039    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:48:55.741442 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.741422    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:48:55.771890 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.771861    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-plugins" pod "68d4b3dd-7845-4ad7-bb11-42caad5c5da9" (UID: "68d4b3dd-7845-4ad7-bb11-42caad5c5da9"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:55.830213 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.830175    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:48:55.830213 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.830202    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/68d4b3dd-7845-4ad7-bb11-42caad5c5da9-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:48:55.902554 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.902523    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:48:55.908090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:55.908067    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:48:56.008622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:56.008550    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38c544ec-f256-40ea-a177-c548c7e716c2" path="/var/lib/kubelet/pods/38c544ec-f256-40ea-a177-c548c7e716c2/volumes"
Oct 30 18:48:56.008986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:56.008972    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68d4b3dd-7845-4ad7-bb11-42caad5c5da9" path="/var/lib/kubelet/pods/68d4b3dd-7845-4ad7-bb11-42caad5c5da9/volumes"
Oct 30 18:48:56.009323 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:56.009304    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d90677fd-8623-42a7-a797-2e2e6d850f2a" path="/var/lib/kubelet/pods/d90677fd-8623-42a7-a797-2e2e6d850f2a/volumes"
Oct 30 18:48:57.551399 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551368    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551631    5746 cpu_manager.go:414] "containerMap: removing container" podUID="d90677fd-8623-42a7-a797-2e2e6d850f2a" containerName="ta-container"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551642    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d90677fd-8623-42a7-a797-2e2e6d850f2a" containerName="ta-container"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551660    5746 cpu_manager.go:414] "containerMap: removing container" podUID="7e0c0381-397e-4d74-876b-932d0d8a126d" containerName="ta-container"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551665    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e0c0381-397e-4d74-876b-932d0d8a126d" containerName="ta-container"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551676    5746 cpu_manager.go:414] "containerMap: removing container" podUID="68d4b3dd-7845-4ad7-bb11-42caad5c5da9" containerName="perses"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551681    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68d4b3dd-7845-4ad7-bb11-42caad5c5da9" containerName="perses"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551690    5746 cpu_manager.go:414] "containerMap: removing container" podUID="38c544ec-f256-40ea-a177-c548c7e716c2" containerName="thanos-querier"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551695    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38c544ec-f256-40ea-a177-c548c7e716c2" containerName="thanos-querier"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551735    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e0c0381-397e-4d74-876b-932d0d8a126d" containerName="ta-container"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551742    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="68d4b3dd-7845-4ad7-bb11-42caad5c5da9" containerName="perses"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551748    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="38c544ec-f256-40ea-a177-c548c7e716c2" containerName="thanos-querier"
Oct 30 18:48:57.551795 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.551757    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="d90677fd-8623-42a7-a797-2e2e6d850f2a" containerName="ta-container"
Oct 30 18:48:57.574562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.574527    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:48:57.574740 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.574656    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:57.589173 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.589151    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:48:57.600766 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.600747    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:48:57.600876 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.600784    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:48:57.646051 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.646027    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/790dc600-ad91-4202-b94f-049ed796609e-data\") pod \"seaweedfs\" (UID: \"790dc600-ad91-4202-b94f-049ed796609e\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:57.646165 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.646055    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k8hg\" (UniqueName: \"kubernetes.io/projected/790dc600-ad91-4202-b94f-049ed796609e-kube-api-access-2k8hg\") pod \"seaweedfs\" (UID: \"790dc600-ad91-4202-b94f-049ed796609e\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:57.747241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.747214    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/790dc600-ad91-4202-b94f-049ed796609e-data\") pod \"seaweedfs\" (UID: \"790dc600-ad91-4202-b94f-049ed796609e\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:57.747389 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.747247    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k8hg\" (UniqueName: \"kubernetes.io/projected/790dc600-ad91-4202-b94f-049ed796609e-kube-api-access-2k8hg\") pod \"seaweedfs\" (UID: \"790dc600-ad91-4202-b94f-049ed796609e\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:57.747600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.747580    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/790dc600-ad91-4202-b94f-049ed796609e-data\") pod \"seaweedfs\" (UID: \"790dc600-ad91-4202-b94f-049ed796609e\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:57.770133 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.770110    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k8hg\" (UniqueName: \"kubernetes.io/projected/790dc600-ad91-4202-b94f-049ed796609e-kube-api-access-2k8hg\") pod \"seaweedfs\" (UID: \"790dc600-ad91-4202-b94f-049ed796609e\") " pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:57.887037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:57.887001    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:58.179108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:58.179024    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:48:58.182969 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:48:58.182940    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod790dc600_ad91_4202_b94f_049ed796609e.slice/crio-6262ab26f2a0adc289300f705931b68c105d122bb6a290f8201b5935ec4c9022 WatchSource:0}: Error finding container 6262ab26f2a0adc289300f705931b68c105d122bb6a290f8201b5935ec4c9022: Status 404 returned error can't find the container with id 6262ab26f2a0adc289300f705931b68c105d122bb6a290f8201b5935ec4c9022
Oct 30 18:48:58.579230 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:58.579144    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/seaweedfs" event={"ID":"790dc600-ad91-4202-b94f-049ed796609e","Type":"ContainerStarted","Data":"59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0"}
Oct 30 18:48:58.579230 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:58.579186    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/seaweedfs" event={"ID":"790dc600-ad91-4202-b94f-049ed796609e","Type":"ContainerStarted","Data":"6262ab26f2a0adc289300f705931b68c105d122bb6a290f8201b5935ec4c9022"}
Oct 30 18:48:58.579688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:58.579236    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:48:58.621781 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:48:58.621732    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/seaweedfs" podStartSLOduration=1.62171911 podStartE2EDuration="1.62171911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:58.621270636 +0000 UTC m=+3108.788088763" watchObservedRunningTime="2026-10-30 18:48:58.62171911 +0000 UTC m=+3108.788537237"
Oct 30 18:49:09.585399 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:09.585370    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:49:12.790219 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:49:12.790180    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:49:19.682139 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.682108    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"]
Oct 30 18:49:19.686428 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.686407    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.712621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.712590    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-ca-bundle\""
Oct 30 18:49:19.712621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.712598    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"test-usage-logs-loki-secret\""
Oct 30 18:49:19.712871 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.712734    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-ingester-grpc\""
Oct 30 18:49:19.712871 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.712740    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-config\""
Oct 30 18:49:19.712871 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.712754    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-ingester-http\""
Oct 30 18:49:19.712871 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.712789    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-dockercfg-6pb2l\""
Oct 30 18:49:19.725166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.725144    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"]
Oct 30 18:49:19.825004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.824970    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-test-usage-logs-loki-secret\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.825004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.825017    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-grpc\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.825277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.825117    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wcbq\" (UniqueName: \"kubernetes.io/projected/d8c141ed-648c-499b-8508-c32f7d4c92a3-kube-api-access-8wcbq\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.825277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.825180    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.825277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.825256    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-http\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.825386 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.825288    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-config\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.926173 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.926136    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.926346 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.926195    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-http\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.926346 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.926229    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-config\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.926346 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.926268    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-test-usage-logs-loki-secret\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.926506 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.926427    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-grpc\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.926506 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.926491    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wcbq\" (UniqueName: \"kubernetes.io/projected/d8c141ed-648c-499b-8508-c32f7d4c92a3-kube-api-access-8wcbq\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.926977 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.926944    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.927118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.927076    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-config\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.929009 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.928986    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-grpc\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.929114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.929021    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-http\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.929114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.929062    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-test-usage-logs-loki-secret\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:19.949978 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:19.949925    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wcbq\" (UniqueName: \"kubernetes.io/projected/d8c141ed-648c-499b-8508-c32f7d4c92a3-kube-api-access-8wcbq\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:21.019208 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.019175    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:49:21.022724 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.022707    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.037824 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.037801    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-index-gateway-grpc\""
Oct 30 18:49:21.049175 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.049151    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-index-gateway-http\""
Oct 30 18:49:21.050003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.049985    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:49:21.136484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.136449    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.136484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.136485    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.136689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.136547    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-test-usage-logs-loki-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.136689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.136583    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cmmf\" (UniqueName: \"kubernetes.io/projected/6752a8e2-455e-4ae7-89ac-4b1d691bc720-kube-api-access-5cmmf\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.136689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.136675    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.136797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.136696    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.238003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.237972    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.238108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.238014    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.238108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.238044    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-test-usage-logs-loki-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.238108 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.238069    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cmmf\" (UniqueName: \"kubernetes.io/projected/6752a8e2-455e-4ae7-89ac-4b1d691bc720-kube-api-access-5cmmf\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.238255 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.238111    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.238255 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.238143    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.238788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.238760    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.239075 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.238842    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.240754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.240729    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.240754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.240746    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-test-usage-logs-loki-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.240884 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.240767    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:21.261666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:21.261645    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cmmf\" (UniqueName: \"kubernetes.io/projected/6752a8e2-455e-4ae7-89ac-4b1d691bc720-kube-api-access-5cmmf\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.272804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.272766    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.272804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.272808    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.273304 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.272852    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.313385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.313353    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") device path: \"\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.313385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.313372    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") device path: \"\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.313385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.313355    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") device path: \"\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.373193 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.373170    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.373342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.373201    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.373342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.373255    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.373342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.373311    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.373443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.373360    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.373443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.373422    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.387579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.387555    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") DevicePath \"csi-77d8f0f59d799d51ff3be9bc71221a143ec00c2335a5e81b4c47b15858001ff3\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.387684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.387627    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") DevicePath \"csi-63fc16ee3224580d120b91f6212f722820785db16c0cdcdd88fdc3f8519dd2ad\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.387684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.387559    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") DevicePath \"csi-8784cb626a5ea44de5a6bb2c861154a9513b88b2561a0b08db835b9cc8529ae9\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.615981 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.615944    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/8d67a4309d5c992166a3eb1f76372dee84e9d17e695a02ef5687339f8e481ef0/globalmount\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.621149 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.621120    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/49ae33b5ff4014339351ebe89a3549f1eae97ccd8a0289bad6d6f86f76fdff92/globalmount\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.640409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.640384    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.644704 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.644680    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.671080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.671045    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/c59e0333bfacae8243fcdbdeca9ad6e70667a5474ab3176a1416861c75db702d/globalmount\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.695268 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.695247    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") pod \"data-science-usage-logs-lokistack-ingester-0\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:25.834074 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.834037    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:25.997794 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:25.997738    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:26.143437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:26.143362    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:49:26.147228 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:49:26.147201    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6752a8e2_455e_4ae7_89ac_4b1d691bc720.slice/crio-12d46a989c9b8f2434ff1acd0cda819590b0f9c9bf3f5356e8e48998c0d04402 WatchSource:0}: Error finding container 12d46a989c9b8f2434ff1acd0cda819590b0f9c9bf3f5356e8e48998c0d04402: Status 404 returned error can't find the container with id 12d46a989c9b8f2434ff1acd0cda819590b0f9c9bf3f5356e8e48998c0d04402
Oct 30 18:49:26.149281 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:26.149266    5746 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:49:26.293451 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:26.293422    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"]
Oct 30 18:49:26.297049 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:49:26.297024    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8c141ed_648c_499b_8508_c32f7d4c92a3.slice/crio-aaca1e93b692120670458235c294fbc112737ca351b3bc68f9b19a60c748948d WatchSource:0}: Error finding container aaca1e93b692120670458235c294fbc112737ca351b3bc68f9b19a60c748948d: Status 404 returned error can't find the container with id aaca1e93b692120670458235c294fbc112737ca351b3bc68f9b19a60c748948d
Oct 30 18:49:26.689688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:26.689653    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" event={"ID":"6752a8e2-455e-4ae7-89ac-4b1d691bc720","Type":"ContainerStarted","Data":"12d46a989c9b8f2434ff1acd0cda819590b0f9c9bf3f5356e8e48998c0d04402"}
Oct 30 18:49:26.690698 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:26.690673    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" event={"ID":"d8c141ed-648c-499b-8508-c32f7d4c92a3","Type":"ContainerStarted","Data":"aaca1e93b692120670458235c294fbc112737ca351b3bc68f9b19a60c748948d"}
Oct 30 18:49:28.701631 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:28.701586    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" event={"ID":"6752a8e2-455e-4ae7-89ac-4b1d691bc720","Type":"ContainerStarted","Data":"6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3"}
Oct 30 18:49:28.702500 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:28.702475    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:49:28.746811 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:28.746748    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" podStartSLOduration=10.320654649 podStartE2EDuration="12.746728269s" totalImagesPullingTime="2.42607362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:28.7461352 +0000 UTC m=+3138.912953326" watchObservedRunningTime="2026-10-30 18:49:28.746728269 +0000 UTC m=+3138.913546405"
Oct 30 18:49:29.707178 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:29.707137    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" event={"ID":"d8c141ed-648c-499b-8508-c32f7d4c92a3","Type":"ContainerStarted","Data":"4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378"}
Oct 30 18:49:29.707677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:29.707339    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:49:29.750456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:29.750406    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" podStartSLOduration=11.144398828 podStartE2EDuration="13.750391637s" totalImagesPullingTime="2.605992809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:29.749855732 +0000 UTC m=+3139.916673858" watchObservedRunningTime="2026-10-30 18:49:29.750391637 +0000 UTC m=+3139.917209767"
Oct 30 18:49:42.789844 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:49:42.789795    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:49:50.717298 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:50.717256    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:49:50.717797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:49:50.717772    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:50:00.716579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:00.716539    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:50:10.715802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:10.715763    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:50:12.790074 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:50:12.790039    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:50:20.717086 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:20.717044    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:50:30.715556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:30.715517    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:50:42.789972 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:50:42.789929    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:50:56.218734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.218695    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"]
Oct 30 18:50:56.222027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.222004    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.248537 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.248477    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-headless-tls\""
Oct 30 18:50:56.248537 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.248497    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-9d8884b6\""
Oct 30 18:50:56.248780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.248660    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-dockercfg-rl5v4\""
Oct 30 18:50:56.249487 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.249463    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"]
Oct 30 18:50:56.331726 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.331693    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otlp-tls-certs\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.331928 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.331748    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otc-internal\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.331928 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.331855    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4hzf\" (UniqueName: \"kubernetes.io/projected/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-kube-api-access-b4hzf\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.432600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.432564    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otlp-tls-certs\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.432796 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.432617    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otc-internal\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.432796 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.432647    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4hzf\" (UniqueName: \"kubernetes.io/projected/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-kube-api-access-b4hzf\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.433383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.433357    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otc-internal\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.435220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.435200    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otlp-tls-certs\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.456305 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.456279    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4hzf\" (UniqueName: \"kubernetes.io/projected/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-kube-api-access-b4hzf\") pod \"data-science-usage-logs-collector-57868947f6-szh4h\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.537118 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.537055    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:50:56.837783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.837706    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"]
Oct 30 18:50:56.841547 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:50:56.841508    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4756a6c_a6d8_49e7_b12e_d861e2ee216d.slice/crio-ecfae95ff0c34499d94e78eb649128ef26857e8daac52ac1afa2c976c938b8d4 WatchSource:0}: Error finding container ecfae95ff0c34499d94e78eb649128ef26857e8daac52ac1afa2c976c938b8d4: Status 404 returned error can't find the container with id ecfae95ff0c34499d94e78eb649128ef26857e8daac52ac1afa2c976c938b8d4
Oct 30 18:50:56.843991 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.843955    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:50:56.844101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:56.844030    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:50:57.072547 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.072509    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h" event={"ID":"b4756a6c-a6d8-49e7-b12e-d861e2ee216d","Type":"ContainerStarted","Data":"d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed"}
Oct 30 18:50:57.072746 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.072560    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h" event={"ID":"b4756a6c-a6d8-49e7-b12e-d861e2ee216d","Type":"ContainerStarted","Data":"ecfae95ff0c34499d94e78eb649128ef26857e8daac52ac1afa2c976c938b8d4"}
Oct 30 18:50:57.116777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.116716    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h" podStartSLOduration=1.116700333 podStartE2EDuration="1.116700333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:57.114517965 +0000 UTC m=+3227.281336111" watchObservedRunningTime="2026-10-30 18:50:57.116700333 +0000 UTC m=+3227.283518456"
Oct 30 18:50:57.596519 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.596448    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"]
Oct 30 18:50:57.600021 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.599990    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.628548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.628519    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ca-bundle\""
Oct 30 18:50:57.628678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.628627    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-dockercfg-h78x9\""
Oct 30 18:50:57.628678 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.628630    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-distributor-http\""
Oct 30 18:50:57.628852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.628714    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-distributor-grpc\""
Oct 30 18:50:57.628852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.628789    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-config\""
Oct 30 18:50:57.634091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.634062    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"]
Oct 30 18:50:57.641061 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.641033    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-http\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.641151 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.641094    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.641216 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.641194    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgjg6\" (UniqueName: \"kubernetes.io/projected/28192d5e-4e09-46fe-ab86-afa16b546160-kube-api-access-hgjg6\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.641257 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.641248    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-grpc\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.641348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.641329    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-config\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.741746 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.741711    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-grpc\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.741934 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.741765    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-config\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.741934 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.741799    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-http\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.741934 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.741842    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.741934 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.741890    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgjg6\" (UniqueName: \"kubernetes.io/projected/28192d5e-4e09-46fe-ab86-afa16b546160-kube-api-access-hgjg6\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.742679 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.742656    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-config\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.742867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.742843    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.744497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.744473    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-grpc\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.744603 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.744472    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-http\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.765048 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.765021    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgjg6\" (UniqueName: \"kubernetes.io/projected/28192d5e-4e09-46fe-ab86-afa16b546160-kube-api-access-hgjg6\") pod \"data-science-lokistack-distributor-b854ccfb4-q696x\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") " pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.776969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.776947    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"]
Oct 30 18:50:57.780616 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.780600    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.804505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.804456    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"test-logs-loki-secret\""
Oct 30 18:50:57.805241 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.805215    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-querier-http\""
Oct 30 18:50:57.805459 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.805429    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-querier-grpc\""
Oct 30 18:50:57.806455 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.806431    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"]
Oct 30 18:50:57.843262 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.843226    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.843429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.843271    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.843429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.843360    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-config\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.843429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.843396    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-test-logs-loki-secret\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.843429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.843417    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.843598 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.843436    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn4z7\" (UniqueName: \"kubernetes.io/projected/bbe061df-8376-4615-9af0-45681fa45cb1-kube-api-access-kn4z7\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.912209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.912127    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:57.945843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.944644    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-test-logs-loki-secret\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.945843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.944693    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.945843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.944724    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn4z7\" (UniqueName: \"kubernetes.io/projected/bbe061df-8376-4615-9af0-45681fa45cb1-kube-api-access-kn4z7\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.945843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.944797    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.945843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.944851    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.945843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.944931    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-config\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.945843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.945795    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-config\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.946356 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.946329    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.948770 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.948743    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-http\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.949235 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.949209    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-test-logs-loki-secret\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.949443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.949422    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-grpc\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:57.971022 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:57.970995    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn4z7\" (UniqueName: \"kubernetes.io/projected/bbe061df-8376-4615-9af0-45681fa45cb1-kube-api-access-kn4z7\") pod \"data-science-lokistack-querier-778ddf848f-shhvs\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") " pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:58.092011 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:58.091981    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:58.213918 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:58.213813    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"]
Oct 30 18:50:58.217880 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:50:58.217847    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28192d5e_4e09_46fe_ab86_afa16b546160.slice/crio-89de43b10ffb516da4ab3ea711e4408fd40c2101e0c590f72a325a97b8a5f12b WatchSource:0}: Error finding container 89de43b10ffb516da4ab3ea711e4408fd40c2101e0c590f72a325a97b8a5f12b: Status 404 returned error can't find the container with id 89de43b10ffb516da4ab3ea711e4408fd40c2101e0c590f72a325a97b8a5f12b
Oct 30 18:50:58.391597 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:58.391558    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"]
Oct 30 18:50:58.395836 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:50:58.395800    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbe061df_8376_4615_9af0_45681fa45cb1.slice/crio-41c61110d954739a3444807b5232e3d85e43fed163db7e9ca3d4aebe01c816f3 WatchSource:0}: Error finding container 41c61110d954739a3444807b5232e3d85e43fed163db7e9ca3d4aebe01c816f3: Status 404 returned error can't find the container with id 41c61110d954739a3444807b5232e3d85e43fed163db7e9ca3d4aebe01c816f3
Oct 30 18:50:59.082315 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:59.082279    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs" event={"ID":"bbe061df-8376-4615-9af0-45681fa45cb1","Type":"ContainerStarted","Data":"dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076"}
Oct 30 18:50:59.082315 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:59.082319    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs" event={"ID":"bbe061df-8376-4615-9af0-45681fa45cb1","Type":"ContainerStarted","Data":"41c61110d954739a3444807b5232e3d85e43fed163db7e9ca3d4aebe01c816f3"}
Oct 30 18:50:59.082836 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:59.082390    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:50:59.083921 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:59.083887    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x" event={"ID":"28192d5e-4e09-46fe-ab86-afa16b546160","Type":"ContainerStarted","Data":"107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1"}
Oct 30 18:50:59.084016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:59.083928    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x" event={"ID":"28192d5e-4e09-46fe-ab86-afa16b546160","Type":"ContainerStarted","Data":"89de43b10ffb516da4ab3ea711e4408fd40c2101e0c590f72a325a97b8a5f12b"}
Oct 30 18:50:59.084060 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:59.084024    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:50:59.124069 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:59.123828    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs" podStartSLOduration=2.123812188 podStartE2EDuration="2.123812188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:59.123198606 +0000 UTC m=+3229.290016733" watchObservedRunningTime="2026-10-30 18:50:59.123812188 +0000 UTC m=+3229.290630315"
Oct 30 18:50:59.164821 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:50:59.164767    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x" podStartSLOduration=2.164752737 podStartE2EDuration="2.164752737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:59.16373116 +0000 UTC m=+3229.330549298" watchObservedRunningTime="2026-10-30 18:50:59.164752737 +0000 UTC m=+3229.331570864"
Oct 30 18:51:00.727687 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.727654    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 18:51:00.731278 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.731260    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.757600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.757564    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ingester-http\""
Oct 30 18:51:00.757773 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.757646    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-lokistack-ingester-grpc\""
Oct 30 18:51:00.758827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.758811    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 18:51:00.873028 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.873001    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.873226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.873036    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgll9\" (UniqueName: \"kubernetes.io/projected/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-kube-api-access-xgll9\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.873226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.873115    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.873226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.873152    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-test-logs-loki-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.873348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.873222    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.873348 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.873276    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.974435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.974395    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.974435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.974436    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgll9\" (UniqueName: \"kubernetes.io/projected/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-kube-api-access-xgll9\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.974662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.974457    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.974662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.974476    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-test-logs-loki-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.974662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.974509    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.974662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.974546    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.975929 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.975872    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ca-bundle\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.983046 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.982984    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ingester-http\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.983208 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.983121    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-config\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.984021 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.983994    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-test-logs-loki-secret\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.984672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.984653    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-data-science-lokistack-ingester-grpc\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:00.999007 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:00.998975    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgll9\" (UniqueName: \"kubernetes.io/projected/516fe8f3-5f12-4635-8c44-ba5c1051dc3e-kube-api-access-xgll9\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:01.785972 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.785937    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-sq2bp"]
Oct 30 18:51:01.790404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.790380    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.805826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.805802    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-logging\"/\"kube-root-ca.crt\""
Oct 30 18:51:01.817600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.817570    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-logging\"/\"openshift-service-ca.crt\""
Oct 30 18:51:01.818003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.817980    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-collector-dockercfg-hnvcg\""
Oct 30 18:51:01.818167 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.818015    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-metrics\""
Oct 30 18:51:01.818167 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.818063    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-config\""
Oct 30 18:51:01.829484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.829455    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-collector-token\""
Oct 30 18:51:01.841958 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.841900    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-trustbundle\""
Oct 30 18:51:01.842960 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.842941    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-sq2bp"]
Oct 30 18:51:01.885474 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885441    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-trusted-ca\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.885736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885481    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config-openshift-service-cacrt\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.885736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885514    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.885736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885649    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sa-token\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-sa-token\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.885736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885676    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgwnc\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-kube-api-access-tgwnc\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.885736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885697    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.885954 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885818    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39d29f26-6727-4895-ab94-738e6c224374-tmp\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.885954 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885853    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"varlogpods\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-varlogpods\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.885954 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885873    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-data-science-cluster-log-forwarder-collector-token\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.886062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.885979    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"datadir\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-datadir\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.886062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.886004    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"entrypoint\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-entrypoint\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987174 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987140    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987191    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sa-token\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-sa-token\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987207    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgwnc\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-kube-api-access-tgwnc\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987222    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987291    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39d29f26-6727-4895-ab94-738e6c224374-tmp\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987385 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:51:01.987316    5746 secret.go:189] Couldn't get secret openshift-logging/data-science-cluster-log-forwarder-metrics: secret "data-science-cluster-log-forwarder-metrics" not found
Oct 30 18:51:01.987385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987342    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"varlogpods\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-varlogpods\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987385 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:51:01.987372    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics podName:39d29f26-6727-4895-ab94-738e6c224374 nodeName:}" failed. No retries permitted until 2026-10-30 18:51:02.487355694 +0000 UTC m=+3232.654173798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics" (UniqueName: "kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics") pod "data-science-cluster-log-forwarder-sq2bp" (UID: "39d29f26-6727-4895-ab94-738e6c224374") : secret "data-science-cluster-log-forwarder-metrics" not found
Oct 30 18:51:01.987385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987385    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-data-science-cluster-log-forwarder-collector-token\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987399    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"varlogpods\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-varlogpods\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987443    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"datadir\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-datadir\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987469    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"entrypoint\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-entrypoint\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987517    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-trusted-ca\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987561    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config-openshift-service-cacrt\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987717    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"datadir\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-datadir\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.987826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.987809    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.988226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.988149    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config-openshift-service-cacrt\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.988226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.988187    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"entrypoint\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-entrypoint\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.988405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.988384    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-trusted-ca\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.989741 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.989719    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39d29f26-6727-4895-ab94-738e6c224374-tmp\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:01.990157 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:01.990134    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-data-science-cluster-log-forwarder-collector-token\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:02.012605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:02.012567    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sa-token\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-sa-token\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:02.012605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:02.012585    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgwnc\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-kube-api-access-tgwnc\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:02.203332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:02.203290    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-sq2bp"]
Oct 30 18:51:02.203595 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:51:02.203571    5746 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp" podUID="39d29f26-6727-4895-ab94-738e6c224374"
Oct 30 18:51:02.492423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:02.492324    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:02.494782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:02.494760    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics\") pod \"data-science-cluster-log-forwarder-sq2bp\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") " pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:03.101383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.101353    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:03.109751 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.109727    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:03.198811 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.198775    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config-openshift-service-cacrt\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.198811 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.198815    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-data-science-cluster-log-forwarder-collector-token\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.198844    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-trusted-ca\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.198946    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.198993    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39d29f26-6727-4895-ab94-738e6c224374-tmp\" (UniqueName: \"kubernetes.io/empty-dir/39d29f26-6727-4895-ab94-738e6c224374-tmp\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199067    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-varlogpods\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-varlogpods\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199096    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-entrypoint\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-entrypoint\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199135    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-kube-api-access-tgwnc\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-kube-api-access-tgwnc\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199160    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-datadir\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-datadir\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199216    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-sa-token\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-sa-token\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199217    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config-openshift-service-cacrt" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "config-openshift-service-cacrt". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:03.199431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199248    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config\") pod \"39d29f26-6727-4895-ab94-738e6c224374\" (UID: \"39d29f26-6727-4895-ab94-738e6c224374\") "
Oct 30 18:51:03.199431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199256    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-datadir" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "datadir". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Oct 30 18:51:03.199431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199314    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-trusted-ca" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:03.199705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199521    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-entrypoint" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "entrypoint". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:03.199705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199568    5746 reconciler_common.go:299] "Volume detached for volume \"config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config-openshift-service-cacrt\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.199705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199661    5746 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-trusted-ca\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.199705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199696    5746 reconciler_common.go:299] "Volume detached for volume \"datadir\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-datadir\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.199903 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199829    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-varlogpods" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "varlogpods". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Oct 30 18:51:03.199998 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.199970    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:03.201371 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.201343    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-data-science-cluster-log-forwarder-collector-token" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "data-science-cluster-log-forwarder-collector-token". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:51:03.201780 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.201760    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "metrics". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:51:03.201844 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.201803    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39d29f26-6727-4895-ab94-738e6c224374-tmp" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:51:03.202016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.201993    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-sa-token" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:03.202016 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.202005    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-kube-api-access-tgwnc" pod "39d29f26-6727-4895-ab94-738e6c224374" (UID: "39d29f26-6727-4895-ab94-738e6c224374"). InnerVolumeSpecName "kube-api-access-tgwnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:03.300638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.300608    5746 reconciler_common.go:299] "Volume detached for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-metrics\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.300638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.300633    5746 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39d29f26-6727-4895-ab94-738e6c224374-tmp\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.300638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.300642    5746 reconciler_common.go:299] "Volume detached for volume \"varlogpods\" (UniqueName: \"kubernetes.io/host-path/39d29f26-6727-4895-ab94-738e6c224374-varlogpods\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.300827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.300651    5746 reconciler_common.go:299] "Volume detached for volume \"entrypoint\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-entrypoint\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.300827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.300660    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgwnc\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-kube-api-access-tgwnc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.300827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.300668    5746 reconciler_common.go:299] "Volume detached for volume \"sa-token\" (UniqueName: \"kubernetes.io/projected/39d29f26-6727-4895-ab94-738e6c224374-sa-token\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.300827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.300676    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/39d29f26-6727-4895-ab94-738e6c224374-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:03.300827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:03.300685    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/39d29f26-6727-4895-ab94-738e6c224374-data-science-cluster-log-forwarder-collector-token\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:51:04.105564 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.105530    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-logging/data-science-cluster-log-forwarder-sq2bp"
Oct 30 18:51:04.197408 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.197375    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-sq2bp"]
Oct 30 18:51:04.202097 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.202062    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-sq2bp"]
Oct 30 18:51:04.217619 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.217579    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-j6fr6"]
Oct 30 18:51:04.222220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.222200    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.242221 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.242186    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-logging\"/\"kube-root-ca.crt\""
Oct 30 18:51:04.242555 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.242404    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-logging\"/\"openshift-service-ca.crt\""
Oct 30 18:51:04.242555 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.242427    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-metrics\""
Oct 30 18:51:04.243302 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.243278    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-j6fr6"]
Oct 30 18:51:04.243302 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.243293    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-collector-dockercfg-hnvcg\""
Oct 30 18:51:04.250747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.250712    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-collector-token\""
Oct 30 18:51:04.253760 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.253735    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-config\""
Oct 30 18:51:04.255280 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.255259    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-logging\"/\"data-science-cluster-log-forwarder-trustbundle\""
Oct 30 18:51:04.308891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.308807    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"varlogpods\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-varlogpods\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.308891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.308851    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fcfd\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-kube-api-access-5fcfd\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.308891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.308877    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config-openshift-service-cacrt\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.309131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.309032    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-metrics\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.309131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.309072    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.309131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.309092    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sa-token\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-sa-token\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.309131 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.309112    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"entrypoint\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-entrypoint\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.309259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.309226    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-trusted-ca\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.309259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.309251    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"datadir\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-datadir\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.309325 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.309274    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35ca760-7dcb-4716-862c-5959034250a4-tmp\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.309359 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.309346    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-data-science-cluster-log-forwarder-collector-token\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.409955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.409889    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sa-token\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-sa-token\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.409961    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"entrypoint\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-entrypoint\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410002    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-trusted-ca\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410027    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"datadir\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-datadir\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410058    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35ca760-7dcb-4716-862c-5959034250a4-tmp\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410102    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-data-science-cluster-log-forwarder-collector-token\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410170    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"varlogpods\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-varlogpods\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410171    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"datadir\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-datadir\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410204    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fcfd\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-kube-api-access-5fcfd\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410235    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config-openshift-service-cacrt\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410275    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-metrics\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410311    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410276    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"varlogpods\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-varlogpods\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.410875 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410849    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"entrypoint\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-entrypoint\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.411037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.410932    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config-openshift-service-cacrt\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.411105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.411030    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.411105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.411030    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-trusted-ca\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.412806 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.412789    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-metrics\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.413130 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.413105    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35ca760-7dcb-4716-862c-5959034250a4-tmp\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.413357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.413337    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-data-science-cluster-log-forwarder-collector-token\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.433373 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.433346    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sa-token\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-sa-token\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.433700 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.433682    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fcfd\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-kube-api-access-5fcfd\") pod \"data-science-cluster-log-forwarder-j6fr6\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") " pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.533938 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.533879    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:51:04.848601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:04.848569    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-j6fr6"]
Oct 30 18:51:04.851650 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:51:04.851619    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35ca760_7dcb_4716_862c_5959034250a4.slice/crio-7ef24c5a76ed323fe7703800b8ee064794dedd0ee38558233ff43ef707040ccc WatchSource:0}: Error finding container 7ef24c5a76ed323fe7703800b8ee064794dedd0ee38558233ff43ef707040ccc: Status 404 returned error can't find the container with id 7ef24c5a76ed323fe7703800b8ee064794dedd0ee38558233ff43ef707040ccc
Oct 30 18:51:05.110656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:05.110576    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6" event={"ID":"f35ca760-7dcb-4716-862c-5959034250a4","Type":"ContainerStarted","Data":"7ef24c5a76ed323fe7703800b8ee064794dedd0ee38558233ff43ef707040ccc"}
Oct 30 18:51:06.012586 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:06.012540    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39d29f26-6727-4895-ab94-738e6c224374" path="/var/lib/kubelet/pods/39d29f26-6727-4895-ab94-738e6c224374/volumes"
Oct 30 18:51:07.746109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.746079    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-e0e89f73-5e44-4995-883c-ac033990affd\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0fc844f63172eda49\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.746590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.746194    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-b2db5575-8704-440b-a541-ffa108189d9c\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019fcd7ae746be804\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.775312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.775280    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-b2db5575-8704-440b-a541-ffa108189d9c\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019fcd7ae746be804\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.775506 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.775336    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-e0e89f73-5e44-4995-883c-ac033990affd\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0fc844f63172eda49\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") device path: \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.847197 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.847150    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-b2db5575-8704-440b-a541-ffa108189d9c\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019fcd7ae746be804\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.847384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.847267    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-e0e89f73-5e44-4995-883c-ac033990affd\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0fc844f63172eda49\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.847384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.847324    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-b2db5575-8704-440b-a541-ffa108189d9c\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019fcd7ae746be804\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.847479 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.847411    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-e0e89f73-5e44-4995-883c-ac033990affd\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0fc844f63172eda49\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.861962 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.861930    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-e0e89f73-5e44-4995-883c-ac033990affd\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0fc844f63172eda49\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") DevicePath \"csi-70858236432dcc7907dfeadb305e881ba0e30c13c23348ba1abb300fb06e9e73\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:07.862143 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:07.861972    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-b2db5575-8704-440b-a541-ffa108189d9c\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019fcd7ae746be804\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") DevicePath \"csi-87a8fa7d8ecca90a16cbf06df8e264803269f890d385c1b2c621322d0a05a752\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:08.097255 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:08.097223    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-b2db5575-8704-440b-a541-ffa108189d9c\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019fcd7ae746be804\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/62c4abc942e4a1be51697d8cdf795b788b276a1f40642e5290244d571884aec3/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:08.097847 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:08.097816    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-e0e89f73-5e44-4995-883c-ac033990affd\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0fc844f63172eda49\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/c7553017f2b8491250e1710b9c540427f2467dfcaeff8aa9c14df6bac34c3632/globalmount\"" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:08.123378 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:08.123350    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-b2db5575-8704-440b-a541-ffa108189d9c\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-019fcd7ae746be804\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:08.123498 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:08.123418    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-e0e89f73-5e44-4995-883c-ac033990affd\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0fc844f63172eda49\") pod \"data-science-lokistack-ingester-0\" (UID: \"516fe8f3-5f12-4635-8c44-ba5c1051dc3e\") " pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:08.242531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:08.242497    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:08.555815 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:08.555788    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 18:51:08.559151 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:51:08.559120    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod516fe8f3_5f12_4635_8c44_ba5c1051dc3e.slice/crio-3a48f5f1ca82510accf161afe56ccb0a1a5ea598ef16adef577ec6a70fc28c8a WatchSource:0}: Error finding container 3a48f5f1ca82510accf161afe56ccb0a1a5ea598ef16adef577ec6a70fc28c8a: Status 404 returned error can't find the container with id 3a48f5f1ca82510accf161afe56ccb0a1a5ea598ef16adef577ec6a70fc28c8a
Oct 30 18:51:09.130399 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:09.130363    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"516fe8f3-5f12-4635-8c44-ba5c1051dc3e","Type":"ContainerStarted","Data":"0543651910aa85d1f42e3008b64c2e44add64fa0db3128ba453489988402e938"}
Oct 30 18:51:09.130399 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:09.130405    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" event={"ID":"516fe8f3-5f12-4635-8c44-ba5c1051dc3e","Type":"ContainerStarted","Data":"3a48f5f1ca82510accf161afe56ccb0a1a5ea598ef16adef577ec6a70fc28c8a"}
Oct 30 18:51:09.130970 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:09.130441    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:51:09.132032 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:09.132011    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6" event={"ID":"f35ca760-7dcb-4716-862c-5959034250a4","Type":"ContainerStarted","Data":"d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b"}
Oct 30 18:51:09.175603 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:09.175559    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podStartSLOduration=12.175547427 podStartE2EDuration="12.175547427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:09.174991188 +0000 UTC m=+3239.341809312" watchObservedRunningTime="2026-10-30 18:51:09.175547427 +0000 UTC m=+3239.342365553"
Oct 30 18:51:09.220850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:09.220802    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6" podStartSLOduration=1.968505065 podStartE2EDuration="5.220787445s" totalImagesPullingTime="3.25228238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:09.219327261 +0000 UTC m=+3239.386145400" watchObservedRunningTime="2026-10-30 18:51:09.220787445 +0000 UTC m=+3239.387605571"
Oct 30 18:51:12.789978 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:51:12.789944    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:51:20.096272 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:20.096194    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:51:20.096272 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:20.096243    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:51:30.141429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:30.141389    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:51:40.139807 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:40.139767    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:51:42.790729 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:51:42.790689    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:51:50.140853 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:51:50.140806    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:52:00.140540 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:00.140512    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:52:10.991520 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:10.991493    5746 scope.go:122] "RemoveContainer" containerID="ef3c8817e40f853b2878d8872a3c26570ea3a9ad51a7fda6fbf50d82cdd809ce"
Oct 30 18:52:10.999529 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:10.999509    5746 scope.go:122] "RemoveContainer" containerID="23380b91c8afe7cf2c25884033a846d9796555b3d57dc9f4343a6aa440f92060"
Oct 30 18:52:11.007285 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:11.007263    5746 scope.go:122] "RemoveContainer" containerID="d04e4e3fa427b48ef038d0daeea4178b8f30a3ad02912f14d7adca1db53553db"
Oct 30 18:52:11.014399 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:11.014384    5746 scope.go:122] "RemoveContainer" containerID="afa726d6eabf8ca25021f59f73fdbd2184897957442ddfa9b567603d3d94d375"
Oct 30 18:52:11.021669 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:11.021650    5746 scope.go:122] "RemoveContainer" containerID="6c549558731430279cb0ae8a785e9cb488a3d4ac9366c8163b5604933972a7e6"
Oct 30 18:52:12.790526 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:12.790489    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:52:21.383450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.383415    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"]
Oct 30 18:52:21.388625 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.388602    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.428574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.428545    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"]
Oct 30 18:52:21.523405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.523374    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e3056171-e8ee-4813-93fd-a8a36069899d-otc-internal\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.523584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.523505    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e3056171-e8ee-4813-93fd-a8a36069899d-otlp-tls-certs\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.523584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.523530    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6llgj\" (UniqueName: \"kubernetes.io/projected/e3056171-e8ee-4813-93fd-a8a36069899d-kube-api-access-6llgj\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.624644 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.624610    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e3056171-e8ee-4813-93fd-a8a36069899d-otc-internal\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.624810 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.624730    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e3056171-e8ee-4813-93fd-a8a36069899d-otlp-tls-certs\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.624810 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.624771    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6llgj\" (UniqueName: \"kubernetes.io/projected/e3056171-e8ee-4813-93fd-a8a36069899d-kube-api-access-6llgj\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.625512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.625476    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e3056171-e8ee-4813-93fd-a8a36069899d-otc-internal\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.627742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.627715    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e3056171-e8ee-4813-93fd-a8a36069899d-otlp-tls-certs\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.651425 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.651354    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6llgj\" (UniqueName: \"kubernetes.io/projected/e3056171-e8ee-4813-93fd-a8a36069899d-kube-api-access-6llgj\") pod \"data-science-usage-logs-collector-6b8fc67d75-c2prv\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.701615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.701579    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:21.712104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.712077    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-ingester-0"]
Oct 30 18:52:21.712464 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.712286    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" containerID="cri-o://0543651910aa85d1f42e3008b64c2e44add64fa0db3128ba453489988402e938" gracePeriod=600
Oct 30 18:52:21.737541 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.737510    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"]
Oct 30 18:52:21.738250 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.737855    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs" podUID="bbe061df-8376-4615-9af0-45681fa45cb1" containerName="loki-querier" containerID="cri-o://dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076" gracePeriod=30
Oct 30 18:52:21.743043 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.743016    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"]
Oct 30 18:52:21.743233 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.743209    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x" podUID="28192d5e-4e09-46fe-ab86-afa16b546160" containerName="loki-distributor" containerID="cri-o://107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1" gracePeriod=30
Oct 30 18:52:21.829277 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.829143    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-j6fr6"]
Oct 30 18:52:21.829463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:21.829371    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6" podUID="f35ca760-7dcb-4716-862c-5959034250a4" containerName="collector" containerID="cri-o://d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b" gracePeriod=10
Oct 30 18:52:22.025420 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.025391    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"]
Oct 30 18:52:22.045169 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.045125    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:52:22.045289 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.045207    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:52:22.096252 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.096217    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:52:22.194560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.194497    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:52:22.231045 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231015    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-data-science-cluster-log-forwarder-collector-token\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231062    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f35ca760-7dcb-4716-862c-5959034250a4-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35ca760-7dcb-4716-862c-5959034250a4-tmp\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231092    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-datadir\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-datadir\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231123    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-kube-api-access-5fcfd\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-kube-api-access-5fcfd\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231146    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-metrics\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-metrics\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231177    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-varlogpods\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-varlogpods\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231247    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231272    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-entrypoint\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-entrypoint\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231296    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-trusted-ca\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231334    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-sa-token\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-sa-token\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231394    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config-openshift-service-cacrt\") pod \"f35ca760-7dcb-4716-862c-5959034250a4\" (UID: \"f35ca760-7dcb-4716-862c-5959034250a4\") "
Oct 30 18:52:22.231534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231471    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-varlogpods" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "varlogpods". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Oct 30 18:52:22.231534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231526    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-datadir" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "datadir". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Oct 30 18:52:22.231978 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.231936    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.232352 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.232325    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config-openshift-service-cacrt" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "config-openshift-service-cacrt". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.232431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.232369    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-trusted-ca" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.232531 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.232515    5746 reconciler_common.go:299] "Volume detached for volume \"datadir\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-datadir\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.232599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.232539    5746 reconciler_common.go:299] "Volume detached for volume \"varlogpods\" (UniqueName: \"kubernetes.io/host-path/f35ca760-7dcb-4716-862c-5959034250a4-varlogpods\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.235299 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.235276    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-kube-api-access-5fcfd" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "kube-api-access-5fcfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:22.236105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.235970    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-sa-token" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:22.236578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.236265    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f35ca760-7dcb-4716-862c-5959034250a4-tmp" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:22.236578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.236390    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-data-science-cluster-log-forwarder-collector-token" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "data-science-cluster-log-forwarder-collector-token". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:22.238589 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.238554    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-entrypoint" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "entrypoint". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.239341 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.239316    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-metrics" pod "f35ca760-7dcb-4716-862c-5959034250a4" (UID: "f35ca760-7dcb-4716-862c-5959034250a4"). InnerVolumeSpecName "metrics". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:22.333306 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333274    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-http\") pod \"bbe061df-8376-4615-9af0-45681fa45cb1\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") "
Oct 30 18:52:22.333480 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333321    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-ca-bundle\") pod \"bbe061df-8376-4615-9af0-45681fa45cb1\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") "
Oct 30 18:52:22.333480 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333395    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-test-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-test-logs-loki-secret\") pod \"bbe061df-8376-4615-9af0-45681fa45cb1\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") "
Oct 30 18:52:22.333480 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333433    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-config\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-config\") pod \"bbe061df-8376-4615-9af0-45681fa45cb1\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") "
Oct 30 18:52:22.333667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333499    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-grpc\") pod \"bbe061df-8376-4615-9af0-45681fa45cb1\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") "
Oct 30 18:52:22.333667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333545    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbe061df-8376-4615-9af0-45681fa45cb1-kube-api-access-kn4z7\" (UniqueName: \"kubernetes.io/projected/bbe061df-8376-4615-9af0-45681fa45cb1-kube-api-access-kn4z7\") pod \"bbe061df-8376-4615-9af0-45681fa45cb1\" (UID: \"bbe061df-8376-4615-9af0-45681fa45cb1\") "
Oct 30 18:52:22.333827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333810    5746 reconciler_common.go:299] "Volume detached for volume \"config-openshift-service-cacrt\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config-openshift-service-cacrt\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.333891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333834    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-cluster-log-forwarder-collector-token\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-data-science-cluster-log-forwarder-collector-token\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.333891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333850    5746 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f35ca760-7dcb-4716-862c-5959034250a4-tmp\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.333891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333864    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fcfd\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-kube-api-access-5fcfd\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.333891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333876    5746 reconciler_common.go:299] "Volume detached for volume \"metrics\" (UniqueName: \"kubernetes.io/secret/f35ca760-7dcb-4716-862c-5959034250a4-metrics\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.333891 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333891    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.334104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333882    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-config" pod "bbe061df-8376-4615-9af0-45681fa45cb1" (UID: "bbe061df-8376-4615-9af0-45681fa45cb1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.334104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333943    5746 reconciler_common.go:299] "Volume detached for volume \"entrypoint\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-entrypoint\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.334104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333956    5746 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f35ca760-7dcb-4716-862c-5959034250a4-trusted-ca\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.334104 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333967    5746 reconciler_common.go:299] "Volume detached for volume \"sa-token\" (UniqueName: \"kubernetes.io/projected/f35ca760-7dcb-4716-862c-5959034250a4-sa-token\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.334478 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.333760    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-ca-bundle" pod "bbe061df-8376-4615-9af0-45681fa45cb1" (UID: "bbe061df-8376-4615-9af0-45681fa45cb1"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.336059 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.335993    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-http" pod "bbe061df-8376-4615-9af0-45681fa45cb1" (UID: "bbe061df-8376-4615-9af0-45681fa45cb1"). InnerVolumeSpecName "data-science-lokistack-querier-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:22.336225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.336068    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe061df-8376-4615-9af0-45681fa45cb1-kube-api-access-kn4z7" pod "bbe061df-8376-4615-9af0-45681fa45cb1" (UID: "bbe061df-8376-4615-9af0-45681fa45cb1"). InnerVolumeSpecName "kube-api-access-kn4z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:22.336225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.336232    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-grpc" pod "bbe061df-8376-4615-9af0-45681fa45cb1" (UID: "bbe061df-8376-4615-9af0-45681fa45cb1"). InnerVolumeSpecName "data-science-lokistack-querier-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:22.336434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.336251    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-test-logs-loki-secret" pod "bbe061df-8376-4615-9af0-45681fa45cb1" (UID: "bbe061df-8376-4615-9af0-45681fa45cb1"). InnerVolumeSpecName "test-logs-loki-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:22.434600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.434574    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn4z7\" (UniqueName: \"kubernetes.io/projected/bbe061df-8376-4615-9af0-45681fa45cb1-kube-api-access-kn4z7\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.434600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.434599    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-http\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.434990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.434610    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.434990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.434621    5746 reconciler_common.go:299] "Volume detached for volume \"test-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-test-logs-loki-secret\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.434990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.434630    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbe061df-8376-4615-9af0-45681fa45cb1-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.434990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.434638    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/bbe061df-8376-4615-9af0-45681fa45cb1-data-science-lokistack-querier-grpc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.443421 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.443396    5746 generic.go:356] "Generic (PLEG): container finished" podID="f35ca760-7dcb-4716-862c-5959034250a4" containerID="d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b" exitCode=0
Oct 30 18:52:22.443550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.443438    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6"
Oct 30 18:52:22.443550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.443485    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6" event={"ID":"f35ca760-7dcb-4716-862c-5959034250a4","Type":"ContainerDied","Data":"d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b"}
Oct 30 18:52:22.443550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.443533    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-logging/data-science-cluster-log-forwarder-j6fr6" event={"ID":"f35ca760-7dcb-4716-862c-5959034250a4","Type":"ContainerDied","Data":"7ef24c5a76ed323fe7703800b8ee064794dedd0ee38558233ff43ef707040ccc"}
Oct 30 18:52:22.443688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.443553    5746 scope.go:122] "RemoveContainer" containerID="d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b"
Oct 30 18:52:22.444969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.444884    5746 generic.go:356] "Generic (PLEG): container finished" podID="bbe061df-8376-4615-9af0-45681fa45cb1" containerID="dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076" exitCode=0
Oct 30 18:52:22.444969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.444944    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"
Oct 30 18:52:22.444969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.444942    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs" event={"ID":"bbe061df-8376-4615-9af0-45681fa45cb1","Type":"ContainerDied","Data":"dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076"}
Oct 30 18:52:22.445167 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.444985    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs" event={"ID":"bbe061df-8376-4615-9af0-45681fa45cb1","Type":"ContainerDied","Data":"41c61110d954739a3444807b5232e3d85e43fed163db7e9ca3d4aebe01c816f3"}
Oct 30 18:52:22.446804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.446785    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv" event={"ID":"e3056171-e8ee-4813-93fd-a8a36069899d","Type":"ContainerStarted","Data":"a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0"}
Oct 30 18:52:22.446920 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.446824    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv" event={"ID":"e3056171-e8ee-4813-93fd-a8a36069899d","Type":"ContainerStarted","Data":"4053f77a3978c04275e720f71509dc8c92aba6ff6f228ee35c75a413a44f3e61"}
Oct 30 18:52:22.474117 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.474101    5746 scope.go:122] "RemoveContainer" containerID="d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b"
Oct 30 18:52:22.474384 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:22.474362    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b\": container with ID starting with d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b not found: ID does not exist" containerID="d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b"
Oct 30 18:52:22.474482 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.474390    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b"} err="failed to get container status \"d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b\": rpc error: code = NotFound desc = could not find container \"d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b\": container with ID starting with d4111bb9b869ae832b48a49930176f59b49d64ee7d81e7bdb00de8b56ec16b1b not found: ID does not exist"
Oct 30 18:52:22.474482 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.474410    5746 scope.go:122] "RemoveContainer" containerID="dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076"
Oct 30 18:52:22.482683 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.482666    5746 scope.go:122] "RemoveContainer" containerID="dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076"
Oct 30 18:52:22.483004 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:22.482979    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076\": container with ID starting with dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076 not found: ID does not exist" containerID="dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076"
Oct 30 18:52:22.483112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.483008    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076"} err="failed to get container status \"dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076\": rpc error: code = NotFound desc = could not find container \"dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076\": container with ID starting with dad3f9bb97487116287c0e839dbf5fb1c07d9e6bf0a1b15d8c2f927aa9d02076 not found: ID does not exist"
Oct 30 18:52:22.513964 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.513870    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv" podStartSLOduration=1.513851114 podStartE2EDuration="1.513851114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:22.512921571 +0000 UTC m=+3312.679739710" watchObservedRunningTime="2026-10-30 18:52:22.513851114 +0000 UTC m=+3312.680669242"
Oct 30 18:52:22.543746 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.543720    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"]
Oct 30 18:52:22.544007 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.543978    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h" podUID="b4756a6c-a6d8-49e7-b12e-d861e2ee216d" containerName="otc-container" containerID="cri-o://d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed" gracePeriod=30
Oct 30 18:52:22.550083 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.550060    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"]
Oct 30 18:52:22.560125 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.560096    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-querier-778ddf848f-shhvs"]
Oct 30 18:52:22.595063 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.595037    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-j6fr6"]
Oct 30 18:52:22.598438 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.598410    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-logging/data-science-cluster-log-forwarder-j6fr6"]
Oct 30 18:52:22.666233 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.666211    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:52:22.792855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.792835    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:52:22.838370 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.838341    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28192d5e-4e09-46fe-ab86-afa16b546160-kube-api-access-hgjg6\" (UniqueName: \"kubernetes.io/projected/28192d5e-4e09-46fe-ab86-afa16b546160-kube-api-access-hgjg6\") pod \"28192d5e-4e09-46fe-ab86-afa16b546160\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") "
Oct 30 18:52:22.838491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.838403    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-config\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-config\") pod \"28192d5e-4e09-46fe-ab86-afa16b546160\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") "
Oct 30 18:52:22.838491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.838439    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-http\") pod \"28192d5e-4e09-46fe-ab86-afa16b546160\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") "
Oct 30 18:52:22.838491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.838474    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-ca-bundle\") pod \"28192d5e-4e09-46fe-ab86-afa16b546160\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") "
Oct 30 18:52:22.838664 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.838539    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-grpc\") pod \"28192d5e-4e09-46fe-ab86-afa16b546160\" (UID: \"28192d5e-4e09-46fe-ab86-afa16b546160\") "
Oct 30 18:52:22.838855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.838824    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-config" pod "28192d5e-4e09-46fe-ab86-afa16b546160" (UID: "28192d5e-4e09-46fe-ab86-afa16b546160"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.839008 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.838945    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-ca-bundle" pod "28192d5e-4e09-46fe-ab86-afa16b546160" (UID: "28192d5e-4e09-46fe-ab86-afa16b546160"). InnerVolumeSpecName "data-science-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.841135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.841105    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28192d5e-4e09-46fe-ab86-afa16b546160-kube-api-access-hgjg6" pod "28192d5e-4e09-46fe-ab86-afa16b546160" (UID: "28192d5e-4e09-46fe-ab86-afa16b546160"). InnerVolumeSpecName "kube-api-access-hgjg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:22.841387 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.841362    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-http" pod "28192d5e-4e09-46fe-ab86-afa16b546160" (UID: "28192d5e-4e09-46fe-ab86-afa16b546160"). InnerVolumeSpecName "data-science-lokistack-distributor-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:22.841612 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.841580    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-grpc" pod "28192d5e-4e09-46fe-ab86-afa16b546160" (UID: "28192d5e-4e09-46fe-ab86-afa16b546160"). InnerVolumeSpecName "data-science-lokistack-distributor-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:22.939279 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939248    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otlp-tls-certs\") pod \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") "
Oct 30 18:52:22.939404 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939330    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otc-internal\" (UniqueName: \"kubernetes.io/configmap/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otc-internal\") pod \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") "
Oct 30 18:52:22.939504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939486    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-kube-api-access-b4hzf\" (UniqueName: \"kubernetes.io/projected/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-kube-api-access-b4hzf\") pod \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\" (UID: \"b4756a6c-a6d8-49e7-b12e-d861e2ee216d\") "
Oct 30 18:52:22.939659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939638    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otc-internal" pod "b4756a6c-a6d8-49e7-b12e-d861e2ee216d" (UID: "b4756a6c-a6d8-49e7-b12e-d861e2ee216d"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:22.939759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939745    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgjg6\" (UniqueName: \"kubernetes.io/projected/28192d5e-4e09-46fe-ab86-afa16b546160-kube-api-access-hgjg6\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.939820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939761    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.939820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939773    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-distributor-http\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-http\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.939820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939783    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.939820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939791    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.939820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.939804    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-lokistack-distributor-grpc\" (UniqueName: \"kubernetes.io/secret/28192d5e-4e09-46fe-ab86-afa16b546160-data-science-lokistack-distributor-grpc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:22.941493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.941473    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otlp-tls-certs" pod "b4756a6c-a6d8-49e7-b12e-d861e2ee216d" (UID: "b4756a6c-a6d8-49e7-b12e-d861e2ee216d"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:22.941581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:22.941496    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-kube-api-access-b4hzf" pod "b4756a6c-a6d8-49e7-b12e-d861e2ee216d" (UID: "b4756a6c-a6d8-49e7-b12e-d861e2ee216d"). InnerVolumeSpecName "kube-api-access-b4hzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:23.040504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.040444    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4hzf\" (UniqueName: \"kubernetes.io/projected/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-kube-api-access-b4hzf\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:23.040504 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.040468    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4756a6c-a6d8-49e7-b12e-d861e2ee216d-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:23.452836 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.452805    5746 generic.go:356] "Generic (PLEG): container finished" podID="28192d5e-4e09-46fe-ab86-afa16b546160" containerID="107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1" exitCode=0
Oct 30 18:52:23.453355 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.452860    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"
Oct 30 18:52:23.453355 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.452893    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x" event={"ID":"28192d5e-4e09-46fe-ab86-afa16b546160","Type":"ContainerDied","Data":"107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1"}
Oct 30 18:52:23.453355 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.452970    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x" event={"ID":"28192d5e-4e09-46fe-ab86-afa16b546160","Type":"ContainerDied","Data":"89de43b10ffb516da4ab3ea711e4408fd40c2101e0c590f72a325a97b8a5f12b"}
Oct 30 18:52:23.453355 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.452997    5746 scope.go:122] "RemoveContainer" containerID="107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1"
Oct 30 18:52:23.454320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.454297    5746 generic.go:356] "Generic (PLEG): container finished" podID="b4756a6c-a6d8-49e7-b12e-d861e2ee216d" containerID="d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed" exitCode=0
Oct 30 18:52:23.454672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.454354    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"
Oct 30 18:52:23.454672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.454367    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h" event={"ID":"b4756a6c-a6d8-49e7-b12e-d861e2ee216d","Type":"ContainerDied","Data":"d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed"}
Oct 30 18:52:23.454672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.454582    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h" event={"ID":"b4756a6c-a6d8-49e7-b12e-d861e2ee216d","Type":"ContainerDied","Data":"ecfae95ff0c34499d94e78eb649128ef26857e8daac52ac1afa2c976c938b8d4"}
Oct 30 18:52:23.462297 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.461939    5746 scope.go:122] "RemoveContainer" containerID="107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1"
Oct 30 18:52:23.462351 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:23.462300    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1\": container with ID starting with 107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1 not found: ID does not exist" containerID="107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1"
Oct 30 18:52:23.462351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.462333    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1"} err="failed to get container status \"107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1\": rpc error: code = NotFound desc = could not find container \"107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1\": container with ID starting with 107cab781f26081fa2055ab84e403eab557cc478cc5b86a0a7ccf51750364fc1 not found: ID does not exist"
Oct 30 18:52:23.462435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.462354    5746 scope.go:122] "RemoveContainer" containerID="d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed"
Oct 30 18:52:23.469951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.469930    5746 scope.go:122] "RemoveContainer" containerID="d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed"
Oct 30 18:52:23.470203 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:23.470183    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed\": container with ID starting with d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed not found: ID does not exist" containerID="d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed"
Oct 30 18:52:23.470271 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.470207    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed"} err="failed to get container status \"d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed\": rpc error: code = NotFound desc = could not find container \"d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed\": container with ID starting with d2710119e950705431792a2475840a4c2f7a9c196205badea76b2b0868f127ed not found: ID does not exist"
Oct 30 18:52:23.503310 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.503285    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"]
Oct 30 18:52:23.509583 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.509562    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-lokistack-distributor-b854ccfb4-q696x"]
Oct 30 18:52:23.543140 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.543116    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"]
Oct 30 18:52:23.548315 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:23.548294    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-57868947f6-szh4h"]
Oct 30 18:52:24.011205 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:24.011171    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28192d5e-4e09-46fe-ab86-afa16b546160" path="/var/lib/kubelet/pods/28192d5e-4e09-46fe-ab86-afa16b546160/volumes"
Oct 30 18:52:24.011544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:24.011531    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4756a6c-a6d8-49e7-b12e-d861e2ee216d" path="/var/lib/kubelet/pods/b4756a6c-a6d8-49e7-b12e-d861e2ee216d/volumes"
Oct 30 18:52:24.011858 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:24.011848    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbe061df-8376-4615-9af0-45681fa45cb1" path="/var/lib/kubelet/pods/bbe061df-8376-4615-9af0-45681fa45cb1/volumes"
Oct 30 18:52:24.012533 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:24.012519    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f35ca760-7dcb-4716-862c-5959034250a4" path="/var/lib/kubelet/pods/f35ca760-7dcb-4716-862c-5959034250a4/volumes"
Oct 30 18:52:30.062360 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.062324    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"]
Oct 30 18:52:30.064921 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.062552    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester" containerID="cri-o://4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378" gracePeriod=600
Oct 30 18:52:30.120985 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.120946    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"]
Oct 30 18:52:30.121376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121334    5746 cpu_manager.go:414] "containerMap: removing container" podUID="bbe061df-8376-4615-9af0-45681fa45cb1" containerName="loki-querier"
Oct 30 18:52:30.121376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121353    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbe061df-8376-4615-9af0-45681fa45cb1" containerName="loki-querier"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121388    5746 cpu_manager.go:414] "containerMap: removing container" podUID="f35ca760-7dcb-4716-862c-5959034250a4" containerName="collector"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121397    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f35ca760-7dcb-4716-862c-5959034250a4" containerName="collector"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121412    5746 cpu_manager.go:414] "containerMap: removing container" podUID="b4756a6c-a6d8-49e7-b12e-d861e2ee216d" containerName="otc-container"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121419    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4756a6c-a6d8-49e7-b12e-d861e2ee216d" containerName="otc-container"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121427    5746 cpu_manager.go:414] "containerMap: removing container" podUID="28192d5e-4e09-46fe-ab86-afa16b546160" containerName="loki-distributor"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121432    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28192d5e-4e09-46fe-ab86-afa16b546160" containerName="loki-distributor"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121497    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbe061df-8376-4615-9af0-45681fa45cb1" containerName="loki-querier"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121508    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="f35ca760-7dcb-4716-862c-5959034250a4" containerName="collector"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121514    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4756a6c-a6d8-49e7-b12e-d861e2ee216d" containerName="otc-container"
Oct 30 18:52:30.121548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.121520    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="28192d5e-4e09-46fe-ab86-afa16b546160" containerName="loki-distributor"
Oct 30 18:52:30.124480 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.124455    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.139067 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.139041    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"test-loki-lifecycle-secret\""
Oct 30 18:52:30.139384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.139367    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-querier-http\""
Oct 30 18:52:30.139384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.139365    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-querier-grpc\""
Oct 30 18:52:30.139946 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.139893    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:52:30.145179 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.145160    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"]
Oct 30 18:52:30.165092 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.165068    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:30.165274 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.165239    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" podUID="6752a8e2-455e-4ae7-89ac-4b1d691bc720" containerName="loki-index-gateway" containerID="cri-o://6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3" gracePeriod=30
Oct 30 18:52:30.204606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.204580    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjkmw\" (UniqueName: \"kubernetes.io/projected/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-kube-api-access-vjkmw\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.204721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.204626    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-http\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.204721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.204650    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.204831 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.204776    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.204922 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.204884    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-grpc\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.204994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.204979    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.306354 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.306313    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-grpc\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.306493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.306373    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.306493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.306413    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkmw\" (UniqueName: \"kubernetes.io/projected/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-kube-api-access-vjkmw\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.306493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.306437    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-http\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.306493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.306462    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.306645 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.306504    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.307301 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.307243    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.307429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.307320    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.308858 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.308837    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-grpc\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.309632 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.309590    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-http\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.309782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.309766    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.330351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.330286    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjkmw\" (UniqueName: \"kubernetes.io/projected/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-kube-api-access-vjkmw\") pod \"data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.422562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.422539    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:30.436561 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.436532    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:30.495453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.494928    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:30.495453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.495500    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" event={"ID":"6752a8e2-455e-4ae7-89ac-4b1d691bc720","Type":"ContainerDied","Data":"6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3"}
Oct 30 18:52:30.495453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.494830    5746 generic.go:356] "Generic (PLEG): container finished" podID="6752a8e2-455e-4ae7-89ac-4b1d691bc720" containerID="6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3" exitCode=0
Oct 30 18:52:30.495453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.495558    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" event={"ID":"6752a8e2-455e-4ae7-89ac-4b1d691bc720","Type":"ContainerDied","Data":"12d46a989c9b8f2434ff1acd0cda819590b0f9c9bf3f5356e8e48998c0d04402"}
Oct 30 18:52:30.495453 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.495588    5746 scope.go:122] "RemoveContainer" containerID="6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3"
Oct 30 18:52:30.507666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.507635    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-http\") pod \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") "
Oct 30 18:52:30.507797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.507707    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6752a8e2-455e-4ae7-89ac-4b1d691bc720-kube-api-access-5cmmf\" (UniqueName: \"kubernetes.io/projected/6752a8e2-455e-4ae7-89ac-4b1d691bc720-kube-api-access-5cmmf\") pod \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") "
Oct 30 18:52:30.507861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.507823    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") "
Oct 30 18:52:30.507954 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.507867    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-ca-bundle\") pod \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") "
Oct 30 18:52:30.507954 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.507900    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-test-usage-logs-loki-secret\") pod \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") "
Oct 30 18:52:30.508068 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.507971    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-config\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-config\") pod \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") "
Oct 30 18:52:30.508068 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.508028    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\" (UID: \"6752a8e2-455e-4ae7-89ac-4b1d691bc720\") "
Oct 30 18:52:30.509973 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.509626    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-ca-bundle" pod "6752a8e2-455e-4ae7-89ac-4b1d691bc720" (UID: "6752a8e2-455e-4ae7-89ac-4b1d691bc720"). InnerVolumeSpecName "data-science-usage-logs-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:30.511214 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.510871    5746 scope.go:122] "RemoveContainer" containerID="6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3"
Oct 30 18:52:30.511949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.511586    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-config" pod "6752a8e2-455e-4ae7-89ac-4b1d691bc720" (UID: "6752a8e2-455e-4ae7-89ac-4b1d691bc720"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:30.511949 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:30.511685    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3\": container with ID starting with 6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3 not found: ID does not exist" containerID="6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3"
Oct 30 18:52:30.511949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.511719    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3"} err="failed to get container status \"6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3\": rpc error: code = NotFound desc = could not find container \"6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3\": container with ID starting with 6a37d97e113345b08264ee4ff9dfe638393942e978d275db2f028ea4a1454bf3 not found: ID does not exist"
Oct 30 18:52:30.512159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.512028    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-grpc" pod "6752a8e2-455e-4ae7-89ac-4b1d691bc720" (UID: "6752a8e2-455e-4ae7-89ac-4b1d691bc720"). InnerVolumeSpecName "data-science-usage-logs-lokistack-index-gateway-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:30.512159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.512070    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-test-usage-logs-loki-secret" pod "6752a8e2-455e-4ae7-89ac-4b1d691bc720" (UID: "6752a8e2-455e-4ae7-89ac-4b1d691bc720"). InnerVolumeSpecName "test-usage-logs-loki-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:30.514009 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.513983    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-http" pod "6752a8e2-455e-4ae7-89ac-4b1d691bc720" (UID: "6752a8e2-455e-4ae7-89ac-4b1d691bc720"). InnerVolumeSpecName "data-science-usage-logs-lokistack-index-gateway-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:30.514734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.514683    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6752a8e2-455e-4ae7-89ac-4b1d691bc720-kube-api-access-5cmmf" pod "6752a8e2-455e-4ae7-89ac-4b1d691bc720" (UID: "6752a8e2-455e-4ae7-89ac-4b1d691bc720"). InnerVolumeSpecName "kube-api-access-5cmmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:30.514959 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.514936    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3" pod "6752a8e2-455e-4ae7-89ac-4b1d691bc720" (UID: "6752a8e2-455e-4ae7-89ac-4b1d691bc720"). InnerVolumeSpecName "pvc-40bb1491-40e8-4bff-a340-7475b8c784f4". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:52:30.609225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.609192    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-http\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:30.609225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.609224    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cmmf\" (UniqueName: \"kubernetes.io/projected/6752a8e2-455e-4ae7-89ac-4b1d691bc720-kube-api-access-5cmmf\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:30.609432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.609255    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:52:30.609432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.609266    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:30.609432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.609276    5746 reconciler_common.go:299] "Volume detached for volume \"test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-test-usage-logs-loki-secret\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:30.609432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.609286    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6752a8e2-455e-4ae7-89ac-4b1d691bc720-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:30.609432 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.609296    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/6752a8e2-455e-4ae7-89ac-4b1d691bc720-data-science-usage-logs-lokistack-index-gateway-grpc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:30.618602 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.618583    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-40bb1491-40e8-4bff-a340-7475b8c784f4" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:52:30.710035 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.710008    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-63fc16ee3224580d120b91f6212f722820785db16c0cdcdd88fdc3f8519dd2ad\""
Oct 30 18:52:30.712120 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.712097    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester" probeResult="failure" output="Get \"https://10.133.0.142:3101/ready\": dial tcp 10.133.0.142:3101: connect: connection refused"
Oct 30 18:52:30.753464 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.753439    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"]
Oct 30 18:52:30.756814 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:52:30.756789    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdc8ea47_213f_4afa_8506_7b6a9cd7a2d5.slice/crio-696d0daadfa1e823b2ea607d6492aaeb03e9013d2fc7074fe92bbe071140b585 WatchSource:0}: Error finding container 696d0daadfa1e823b2ea607d6492aaeb03e9013d2fc7074fe92bbe071140b585: Status 404 returned error can't find the container with id 696d0daadfa1e823b2ea607d6492aaeb03e9013d2fc7074fe92bbe071140b585
Oct 30 18:52:30.858605 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.858569    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:30.869466 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.869368    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:30.896044 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.896016    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:52:30.914283 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.914256    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:30.914665 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.914648    5746 cpu_manager.go:414] "containerMap: removing container" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester"
Oct 30 18:52:30.914735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.914668    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester"
Oct 30 18:52:30.914735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.914718    5746 cpu_manager.go:414] "containerMap: removing container" podUID="6752a8e2-455e-4ae7-89ac-4b1d691bc720" containerName="loki-index-gateway"
Oct 30 18:52:30.914735 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.914726    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6752a8e2-455e-4ae7-89ac-4b1d691bc720" containerName="loki-index-gateway"
Oct 30 18:52:30.914850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.914816    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="6752a8e2-455e-4ae7-89ac-4b1d691bc720" containerName="loki-index-gateway"
Oct 30 18:52:30.914850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.914829    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerName="loki-ingester"
Oct 30 18:52:30.917963 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.917944    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:30.934935 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.934893    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-index-gateway-grpc\""
Oct 30 18:52:30.934935 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.934897    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-index-gateway-http\""
Oct 30 18:52:30.940258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:30.940239    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:31.013425 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013392    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-test-usage-logs-loki-secret\") pod \"d8c141ed-648c-499b-8508-c32f7d4c92a3\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") "
Oct 30 18:52:31.013703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013440    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-http\") pod \"d8c141ed-648c-499b-8508-c32f7d4c92a3\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") "
Oct 30 18:52:31.013703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013536    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") pod \"d8c141ed-648c-499b-8508-c32f7d4c92a3\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") "
Oct 30 18:52:31.013703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013568    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-grpc\") pod \"d8c141ed-648c-499b-8508-c32f7d4c92a3\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") "
Oct 30 18:52:31.013703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013606    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8c141ed-648c-499b-8508-c32f7d4c92a3-kube-api-access-8wcbq\" (UniqueName: \"kubernetes.io/projected/d8c141ed-648c-499b-8508-c32f7d4c92a3-kube-api-access-8wcbq\") pod \"d8c141ed-648c-499b-8508-c32f7d4c92a3\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") "
Oct 30 18:52:31.013703 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013634    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-config\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-config\") pod \"d8c141ed-648c-499b-8508-c32f7d4c92a3\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") "
Oct 30 18:52:31.014020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013718    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ca-bundle\") pod \"d8c141ed-648c-499b-8508-c32f7d4c92a3\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") "
Oct 30 18:52:31.014020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013810    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") pod \"d8c141ed-648c-499b-8508-c32f7d4c92a3\" (UID: \"d8c141ed-648c-499b-8508-c32f7d4c92a3\") "
Oct 30 18:52:31.014020 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.013963    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.014176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.014017    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.014176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.014124    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.014176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.014158    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.014339 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.014190    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gnwl\" (UniqueName: \"kubernetes.io/projected/22047218-41e3-403d-a2ad-6c926fce40c8-kube-api-access-7gnwl\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.014339 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.014258    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.014489 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.014456    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ca-bundle" pod "d8c141ed-648c-499b-8508-c32f7d4c92a3" (UID: "d8c141ed-648c-499b-8508-c32f7d4c92a3"). InnerVolumeSpecName "data-science-usage-logs-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:31.014489 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.014468    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-config" pod "d8c141ed-648c-499b-8508-c32f7d4c92a3" (UID: "d8c141ed-648c-499b-8508-c32f7d4c92a3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:31.016415 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.016384    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-grpc" pod "d8c141ed-648c-499b-8508-c32f7d4c92a3" (UID: "d8c141ed-648c-499b-8508-c32f7d4c92a3"). InnerVolumeSpecName "data-science-usage-logs-lokistack-ingester-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:31.016544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.016454    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-test-usage-logs-loki-secret" pod "d8c141ed-648c-499b-8508-c32f7d4c92a3" (UID: "d8c141ed-648c-499b-8508-c32f7d4c92a3"). InnerVolumeSpecName "test-usage-logs-loki-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:31.016714 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.016542    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-http" pod "d8c141ed-648c-499b-8508-c32f7d4c92a3" (UID: "d8c141ed-648c-499b-8508-c32f7d4c92a3"). InnerVolumeSpecName "data-science-usage-logs-lokistack-ingester-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:31.016762 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.016688    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c141ed-648c-499b-8508-c32f7d4c92a3-kube-api-access-8wcbq" pod "d8c141ed-648c-499b-8508-c32f7d4c92a3" (UID: "d8c141ed-648c-499b-8508-c32f7d4c92a3"). InnerVolumeSpecName "kube-api-access-8wcbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:31.016864 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.016848    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac" pod "d8c141ed-648c-499b-8508-c32f7d4c92a3" (UID: "d8c141ed-648c-499b-8508-c32f7d4c92a3"). InnerVolumeSpecName "pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:52:31.017279 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.017261    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7" pod "d8c141ed-648c-499b-8508-c32f7d4c92a3" (UID: "d8c141ed-648c-499b-8508-c32f7d4c92a3"). InnerVolumeSpecName "pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:52:31.114643 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.114608    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.114643 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.114645    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.114665    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gnwl\" (UniqueName: \"kubernetes.io/projected/22047218-41e3-403d-a2ad-6c926fce40c8-kube-api-access-7gnwl\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.114709    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.114745    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.114787    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.114950    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.114984    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.115000    5746 reconciler_common.go:299] "Volume detached for volume \"test-usage-logs-loki-secret\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-test-usage-logs-loki-secret\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.115018    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-ingester-http\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-http\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.115039    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.115053    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-ingester-grpc\" (UniqueName: \"kubernetes.io/secret/d8c141ed-648c-499b-8508-c32f7d4c92a3-data-science-usage-logs-lokistack-ingester-grpc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.115082    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wcbq\" (UniqueName: \"kubernetes.io/projected/d8c141ed-648c-499b-8508-c32f7d4c92a3-kube-api-access-8wcbq\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:31.115187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.115098    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8c141ed-648c-499b-8508-c32f7d4c92a3-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:31.115830 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.115550    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.116482 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.116452    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.118076 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.118056    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.118357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.118341    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.118986 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.118965    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.124813 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.124799    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:52:31.126866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.126850    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:52:31.138324 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.138305    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gnwl\" (UniqueName: \"kubernetes.io/projected/22047218-41e3-403d-a2ad-6c926fce40c8-kube-api-access-7gnwl\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:31.216445 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.216410    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-f16319de-02a0-44df-9fbf-4a175d4ad3b9\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-026b4460e5abf51ac\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-77d8f0f59d799d51ff3be9bc71221a143ec00c2335a5e81b4c47b15858001ff3\""
Oct 30 18:52:31.216580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.216450    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-9bd50dcf-f652-43c4-bf16-dd7f49cb7f7e\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03b132b0a533702b7\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-8784cb626a5ea44de5a6bb2c861154a9513b88b2561a0b08db835b9cc8529ae9\""
Oct 30 18:52:31.416793 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.416716    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"]
Oct 30 18:52:31.421358 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.421334    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.435783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.435756    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-compactor-http\""
Oct 30 18:52:31.435783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.435756    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-compactor-grpc\""
Oct 30 18:52:31.440722 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.440701    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"]
Oct 30 18:52:31.500856 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.500824    5746 generic.go:356] "Generic (PLEG): container finished" podID="d8c141ed-648c-499b-8508-c32f7d4c92a3" containerID="4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378" exitCode=0
Oct 30 18:52:31.501034 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.500883    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"
Oct 30 18:52:31.501034 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.500893    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" event={"ID":"d8c141ed-648c-499b-8508-c32f7d4c92a3","Type":"ContainerDied","Data":"4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378"}
Oct 30 18:52:31.501034 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.500962    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0" event={"ID":"d8c141ed-648c-499b-8508-c32f7d4c92a3","Type":"ContainerDied","Data":"aaca1e93b692120670458235c294fbc112737ca351b3bc68f9b19a60c748948d"}
Oct 30 18:52:31.501034 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.500988    5746 scope.go:122] "RemoveContainer" containerID="4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378"
Oct 30 18:52:31.502663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.502636    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" event={"ID":"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5","Type":"ContainerStarted","Data":"2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a"}
Oct 30 18:52:31.502782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.502676    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" event={"ID":"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5","Type":"ContainerStarted","Data":"696d0daadfa1e823b2ea607d6492aaeb03e9013d2fc7074fe92bbe071140b585"}
Oct 30 18:52:31.502782 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.502703    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:31.511483 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.511454    5746 scope.go:122] "RemoveContainer" containerID="4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378"
Oct 30 18:52:31.511780 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:31.511759    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378\": container with ID starting with 4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378 not found: ID does not exist" containerID="4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378"
Oct 30 18:52:31.511880 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.511786    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378"} err="failed to get container status \"4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378\": rpc error: code = NotFound desc = could not find container \"4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378\": container with ID starting with 4701d491fb7f2a258951bbf4dcf130fca6cbac0005428ad30ab85a190f3ba378 not found: ID does not exist"
Oct 30 18:52:31.518488 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.518460    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.518666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.518500    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-config\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.518666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.518520    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.518666 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.518554    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-grpc\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.518809 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.518661    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wg47\" (UniqueName: \"kubernetes.io/projected/ab448805-f34e-4d67-a617-3d4c26b3c72d-kube-api-access-4wg47\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.518809 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.518709    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-http\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.549695 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.549673    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"]
Oct 30 18:52:31.555594 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.555561    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-ingester-0"]
Oct 30 18:52:31.597147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.597093    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" podStartSLOduration=1.59707526 podStartE2EDuration="1.59707526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:31.596359782 +0000 UTC m=+3321.763177921" watchObservedRunningTime="2026-10-30 18:52:31.59707526 +0000 UTC m=+3321.763893388"
Oct 30 18:52:31.619758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.619729    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.619889 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.619785    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-config\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.619889 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.619812    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.619889 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.619848    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-grpc\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.620089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.619930    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wg47\" (UniqueName: \"kubernetes.io/projected/ab448805-f34e-4d67-a617-3d4c26b3c72d-kube-api-access-4wg47\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.620089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.619973    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-http\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.621018 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.620966    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.621018 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.621008    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-config\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.623050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.623010    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-grpc\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.623158 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.623137    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.623704 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.623681    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-http\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:31.647689 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:31.647669    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wg47\" (UniqueName: \"kubernetes.io/projected/ab448805-f34e-4d67-a617-3d4c26b3c72d-kube-api-access-4wg47\") pod \"data-science-usage-logs-lokistack-compactor-0\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:32.008845 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.008812    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6752a8e2-455e-4ae7-89ac-4b1d691bc720" path="/var/lib/kubelet/pods/6752a8e2-455e-4ae7-89ac-4b1d691bc720/volumes"
Oct 30 18:52:32.009270 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.009256    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8c141ed-648c-499b-8508-c32f7d4c92a3" path="/var/lib/kubelet/pods/d8c141ed-648c-499b-8508-c32f7d4c92a3/volumes"
Oct 30 18:52:32.395820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.395778    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"]
Oct 30 18:52:32.396322 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.396021    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv" podUID="e3056171-e8ee-4813-93fd-a8a36069899d" containerName="otc-container" containerID="cri-o://a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0" gracePeriod=30
Oct 30 18:52:32.627400 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:32.627196    5746 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-usage-logs-lokistack-config: configmap "data-science-usage-logs-lokistack-config" not found
Oct 30 18:52:32.627400 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:32.627269    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config podName:fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5 nodeName:}" failed. No retries permitted until 2026-10-30 18:52:33.127247544 +0000 UTC m=+3323.294065670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config") pod "data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5") : configmap "data-science-usage-logs-lokistack-config" not found
Oct 30 18:52:32.665239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.665210    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:32.728439 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.728408    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3056171-e8ee-4813-93fd-a8a36069899d-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e3056171-e8ee-4813-93fd-a8a36069899d-otlp-tls-certs\") pod \"e3056171-e8ee-4813-93fd-a8a36069899d\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") "
Oct 30 18:52:32.728631 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.728448    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3056171-e8ee-4813-93fd-a8a36069899d-kube-api-access-6llgj\" (UniqueName: \"kubernetes.io/projected/e3056171-e8ee-4813-93fd-a8a36069899d-kube-api-access-6llgj\") pod \"e3056171-e8ee-4813-93fd-a8a36069899d\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") "
Oct 30 18:52:32.728631 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.728544    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3056171-e8ee-4813-93fd-a8a36069899d-otc-internal\" (UniqueName: \"kubernetes.io/configmap/e3056171-e8ee-4813-93fd-a8a36069899d-otc-internal\") pod \"e3056171-e8ee-4813-93fd-a8a36069899d\" (UID: \"e3056171-e8ee-4813-93fd-a8a36069899d\") "
Oct 30 18:52:32.728970 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.728941    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3056171-e8ee-4813-93fd-a8a36069899d-otc-internal" pod "e3056171-e8ee-4813-93fd-a8a36069899d" (UID: "e3056171-e8ee-4813-93fd-a8a36069899d"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:32.730839 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.730815    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3056171-e8ee-4813-93fd-a8a36069899d-kube-api-access-6llgj" pod "e3056171-e8ee-4813-93fd-a8a36069899d" (UID: "e3056171-e8ee-4813-93fd-a8a36069899d"). InnerVolumeSpecName "kube-api-access-6llgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:32.731135 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.731117    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3056171-e8ee-4813-93fd-a8a36069899d-otlp-tls-certs" pod "e3056171-e8ee-4813-93fd-a8a36069899d" (UID: "e3056171-e8ee-4813-93fd-a8a36069899d"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:32.741085 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.741056    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:32.759182 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:32.759152    5746 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[storage], unattached volumes=[storage], failed to process volumes=[]: context canceled" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" podUID="22047218-41e3-403d-a2ad-6c926fce40c8"
Oct 30 18:52:32.789033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.789005    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"]
Oct 30 18:52:32.802351 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:32.802319    5746 csi_plugin.go:220] "Failed to get volume attachment" err="volumeattachments.storage.k8s.io \"csi-574d1d46a57b20e0d08714c0638e4988c128e30bcf504b6704e89e9dc3f50d32\" is forbidden: User \"system:node:ip-10-0-131-31.ec2.internal\" cannot get resource \"volumeattachments\" in API group \"storage.k8s.io\" at the cluster scope: no relationship found between node 'ip-10-0-131-31.ec2.internal' and this object" attachmentName="csi-574d1d46a57b20e0d08714c0638e4988c128e30bcf504b6704e89e9dc3f50d32"
Oct 30 18:52:32.802501 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:32.802366    5746 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[storage], unattached volumes=[storage], failed to process volumes=[]: context canceled" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0" podUID="ab448805-f34e-4d67-a617-3d4c26b3c72d"
Oct 30 18:52:32.809240 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.809219    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"]
Oct 30 18:52:32.829886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.829863    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/e3056171-e8ee-4813-93fd-a8a36069899d-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:32.829886 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.829887    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/e3056171-e8ee-4813-93fd-a8a36069899d-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:32.830036 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:32.829897    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6llgj\" (UniqueName: \"kubernetes.io/projected/e3056171-e8ee-4813-93fd-a8a36069899d-kube-api-access-6llgj\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.133350 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:33.133320    5746 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-usage-logs-lokistack-config: configmap "data-science-usage-logs-lokistack-config" not found
Oct 30 18:52:33.133524 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:33.133388    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config podName:fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5 nodeName:}" failed. No retries permitted until 2026-10-30 18:52:34.133373557 +0000 UTC m=+3324.300191663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config") pod "data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5") : configmap "data-science-usage-logs-lokistack-config" not found
Oct 30 18:52:33.518951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.518851    5746 generic.go:356] "Generic (PLEG): container finished" podID="e3056171-e8ee-4813-93fd-a8a36069899d" containerID="a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0" exitCode=0
Oct 30 18:52:33.518951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.518926    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"
Oct 30 18:52:33.519492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.518961    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv" event={"ID":"e3056171-e8ee-4813-93fd-a8a36069899d","Type":"ContainerDied","Data":"a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0"}
Oct 30 18:52:33.519492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.519009    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv" event={"ID":"e3056171-e8ee-4813-93fd-a8a36069899d","Type":"ContainerDied","Data":"4053f77a3978c04275e720f71509dc8c92aba6ff6f228ee35c75a413a44f3e61"}
Oct 30 18:52:33.519492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.519029    5746 scope.go:122] "RemoveContainer" containerID="a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0"
Oct 30 18:52:33.519492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.519109    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:33.519492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.519344    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:33.519771 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.519542    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" podUID="fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" containerName="loki-querier" containerID="cri-o://2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a" gracePeriod=30
Oct 30 18:52:33.526959 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.526935    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:33.528459 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.528447    5746 scope.go:122] "RemoveContainer" containerID="a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0"
Oct 30 18:52:33.528801 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:33.528740    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0\": container with ID starting with a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0 not found: ID does not exist" containerID="a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0"
Oct 30 18:52:33.528801 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.528773    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0"} err="failed to get container status \"a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0\": rpc error: code = NotFound desc = could not find container \"a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0\": container with ID starting with a22ffae80fa4b7514ea80665efb44cccd153a02ac5717d3bd15abf09e9c230f0 not found: ID does not exist"
Oct 30 18:52:33.532275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.532256    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:33.637378 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637350    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-config\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-config\") pod \"22047218-41e3-403d-a2ad-6c926fce40c8\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") "
Oct 30 18:52:33.637556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637386    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-test-loki-lifecycle-secret\") pod \"ab448805-f34e-4d67-a617-3d4c26b3c72d\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") "
Oct 30 18:52:33.637556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637405    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab448805-f34e-4d67-a617-3d4c26b3c72d-kube-api-access-4wg47\" (UniqueName: \"kubernetes.io/projected/ab448805-f34e-4d67-a617-3d4c26b3c72d-kube-api-access-4wg47\") pod \"ab448805-f34e-4d67-a617-3d4c26b3c72d\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") "
Oct 30 18:52:33.637556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637434    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-ca-bundle\") pod \"ab448805-f34e-4d67-a617-3d4c26b3c72d\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") "
Oct 30 18:52:33.637556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637462    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-test-loki-lifecycle-secret\") pod \"22047218-41e3-403d-a2ad-6c926fce40c8\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") "
Oct 30 18:52:33.637556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637488    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-http\") pod \"22047218-41e3-403d-a2ad-6c926fce40c8\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") "
Oct 30 18:52:33.637556 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637526    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22047218-41e3-403d-a2ad-6c926fce40c8-kube-api-access-7gnwl\" (UniqueName: \"kubernetes.io/projected/22047218-41e3-403d-a2ad-6c926fce40c8-kube-api-access-7gnwl\") pod \"22047218-41e3-403d-a2ad-6c926fce40c8\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") "
Oct 30 18:52:33.637898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637563    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-http\") pod \"ab448805-f34e-4d67-a617-3d4c26b3c72d\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") "
Oct 30 18:52:33.637898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637623    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-config\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-config\") pod \"ab448805-f34e-4d67-a617-3d4c26b3c72d\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") "
Oct 30 18:52:33.637898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637671    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-ca-bundle\") pod \"22047218-41e3-403d-a2ad-6c926fce40c8\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") "
Oct 30 18:52:33.637898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637707    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"22047218-41e3-403d-a2ad-6c926fce40c8\" (UID: \"22047218-41e3-403d-a2ad-6c926fce40c8\") "
Oct 30 18:52:33.637898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637794    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-grpc\") pod \"ab448805-f34e-4d67-a617-3d4c26b3c72d\" (UID: \"ab448805-f34e-4d67-a617-3d4c26b3c72d\") "
Oct 30 18:52:33.637898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637830    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-config" pod "22047218-41e3-403d-a2ad-6c926fce40c8" (UID: "22047218-41e3-403d-a2ad-6c926fce40c8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.637898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.637863    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-ca-bundle" pod "ab448805-f34e-4d67-a617-3d4c26b3c72d" (UID: "ab448805-f34e-4d67-a617-3d4c26b3c72d"). InnerVolumeSpecName "data-science-usage-logs-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.638320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.638128    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.638320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.638152    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.638320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.638233    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-ca-bundle" pod "22047218-41e3-403d-a2ad-6c926fce40c8" (UID: "22047218-41e3-403d-a2ad-6c926fce40c8"). InnerVolumeSpecName "data-science-usage-logs-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.638739 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.638711    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-config" pod "ab448805-f34e-4d67-a617-3d4c26b3c72d" (UID: "ab448805-f34e-4d67-a617-3d4c26b3c72d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.640424 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.640391    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab448805-f34e-4d67-a617-3d4c26b3c72d-kube-api-access-4wg47" pod "ab448805-f34e-4d67-a617-3d4c26b3c72d" (UID: "ab448805-f34e-4d67-a617-3d4c26b3c72d"). InnerVolumeSpecName "kube-api-access-4wg47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.640618 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.640595    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-test-loki-lifecycle-secret" pod "ab448805-f34e-4d67-a617-3d4c26b3c72d" (UID: "ab448805-f34e-4d67-a617-3d4c26b3c72d"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:33.640898 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.640878    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-grpc" pod "ab448805-f34e-4d67-a617-3d4c26b3c72d" (UID: "ab448805-f34e-4d67-a617-3d4c26b3c72d"). InnerVolumeSpecName "data-science-usage-logs-lokistack-compactor-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:33.640991 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.640928    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-grpc" pod "22047218-41e3-403d-a2ad-6c926fce40c8" (UID: "22047218-41e3-403d-a2ad-6c926fce40c8"). InnerVolumeSpecName "data-science-usage-logs-lokistack-index-gateway-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:33.641305 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.641280    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-test-loki-lifecycle-secret" pod "22047218-41e3-403d-a2ad-6c926fce40c8" (UID: "22047218-41e3-403d-a2ad-6c926fce40c8"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:33.641423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.641401    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-http" pod "ab448805-f34e-4d67-a617-3d4c26b3c72d" (UID: "ab448805-f34e-4d67-a617-3d4c26b3c72d"). InnerVolumeSpecName "data-science-usage-logs-lokistack-compactor-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:33.641481 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.641454    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22047218-41e3-403d-a2ad-6c926fce40c8-kube-api-access-7gnwl" pod "22047218-41e3-403d-a2ad-6c926fce40c8" (UID: "22047218-41e3-403d-a2ad-6c926fce40c8"). InnerVolumeSpecName "kube-api-access-7gnwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.641549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.641533    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-http" pod "22047218-41e3-403d-a2ad-6c926fce40c8" (UID: "22047218-41e3-403d-a2ad-6c926fce40c8"). InnerVolumeSpecName "data-science-usage-logs-lokistack-index-gateway-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:33.650262 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.650242    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"]
Oct 30 18:52:33.654671 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.654649    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-6b8fc67d75-c2prv"]
Oct 30 18:52:33.738969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.738944    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-compactor-grpc\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-grpc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.738969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.738968    5746 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-test-loki-lifecycle-secret\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.739110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.738978    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wg47\" (UniqueName: \"kubernetes.io/projected/ab448805-f34e-4d67-a617-3d4c26b3c72d-kube-api-access-4wg47\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.739110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.738988    5746 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-test-loki-lifecycle-secret\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.739110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.738997    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-http\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.739110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.739007    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gnwl\" (UniqueName: \"kubernetes.io/projected/22047218-41e3-403d-a2ad-6c926fce40c8-kube-api-access-7gnwl\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.739110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.739015    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-compactor-http\" (UniqueName: \"kubernetes.io/secret/ab448805-f34e-4d67-a617-3d4c26b3c72d-data-science-usage-logs-lokistack-compactor-http\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.739110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.739024    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab448805-f34e-4d67-a617-3d4c26b3c72d-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.739110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.739032    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.739110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:33.739042    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/22047218-41e3-403d-a2ad-6c926fce40c8-data-science-usage-logs-lokistack-index-gateway-grpc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:34.008452 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:34.008426    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3056171-e8ee-4813-93fd-a8a36069899d" path="/var/lib/kubelet/pods/e3056171-e8ee-4813-93fd-a8a36069899d/volumes"
Oct 30 18:52:34.143316 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:34.143286    5746 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-usage-logs-lokistack-config: configmap "data-science-usage-logs-lokistack-config" not found
Oct 30 18:52:34.143489 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:34.143355    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config podName:fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5 nodeName:}" failed. No retries permitted until 2026-10-30 18:52:36.143337951 +0000 UTC m=+3326.310156056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config") pod "data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5") : configmap "data-science-usage-logs-lokistack-config" not found
Oct 30 18:52:34.524663 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:34.524630    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"
Oct 30 18:52:34.525101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:34.524630    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:34.614361 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:34.614323    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:34.617920 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:34.617882    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:34.685980 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:34.685952    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"]
Oct 30 18:52:34.690260 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:34.690236    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-compactor-0"]
Oct 30 18:52:36.013951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:36.013897    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22047218-41e3-403d-a2ad-6c926fce40c8" path="/var/lib/kubelet/pods/22047218-41e3-403d-a2ad-6c926fce40c8/volumes"
Oct 30 18:52:36.014340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:36.014197    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab448805-f34e-4d67-a617-3d4c26b3c72d" path="/var/lib/kubelet/pods/ab448805-f34e-4d67-a617-3d4c26b3c72d/volumes"
Oct 30 18:52:36.159266 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:36.159240    5746 configmap.go:193] Couldn't get configMap redhat-ods-monitoring/data-science-usage-logs-lokistack-config: configmap "data-science-usage-logs-lokistack-config" not found
Oct 30 18:52:36.159423 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:36.159327    5746 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config podName:fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5 nodeName:}" failed. No retries permitted until 2026-10-30 18:52:40.159311962 +0000 UTC m=+3330.326130080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config") pod "data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5") : configmap "data-science-usage-logs-lokistack-config" not found
Oct 30 18:52:38.773994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.773958    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:38.774505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.774354    5746 cpu_manager.go:414] "containerMap: removing container" podUID="e3056171-e8ee-4813-93fd-a8a36069899d" containerName="otc-container"
Oct 30 18:52:38.774505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.774371    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3056171-e8ee-4813-93fd-a8a36069899d" containerName="otc-container"
Oct 30 18:52:38.774505 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.774476    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3056171-e8ee-4813-93fd-a8a36069899d" containerName="otc-container"
Oct 30 18:52:38.778818 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.778796    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.798223 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.798200    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:38.838052 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.838033    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-index-gateway-grpc\""
Oct 30 18:52:38.838052 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.838043    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-index-gateway-http\""
Oct 30 18:52:38.838221 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.838043    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-lokistack-dockercfg-hb59n\""
Oct 30 18:52:38.883860 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.883831    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.884017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.883873    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.884017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.883893    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.884141 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.884033    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.884141 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.884084    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxvlh\" (UniqueName: \"kubernetes.io/projected/1234b4ef-79ca-43db-98cf-00dd8c75fb85-kube-api-access-dxvlh\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.884286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.884163    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.985246 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.985201    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.985437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.985250    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxvlh\" (UniqueName: \"kubernetes.io/projected/1234b4ef-79ca-43db-98cf-00dd8c75fb85-kube-api-access-dxvlh\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.985437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.985292    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.985437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.985344    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.985437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.985397    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.985437 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.985421    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.986158 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.986124    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-config\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.986336 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.986289    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-ca-bundle\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.987956 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.987927    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-http\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.988040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.987963    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:38.988089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:38.988035    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-test-loki-lifecycle-secret\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.012359 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.012336    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxvlh\" (UniqueName: \"kubernetes.io/projected/1234b4ef-79ca-43db-98cf-00dd8c75fb85-kube-api-access-dxvlh\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.173376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.173354    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:39.187837 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.187808    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.216615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.216584    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") device path: \"\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.288834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.288756    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-ca-bundle\") pod \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") "
Oct 30 18:52:39.288834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.288802    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-kube-api-access-vjkmw\" (UniqueName: \"kubernetes.io/projected/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-kube-api-access-vjkmw\") pod \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") "
Oct 30 18:52:39.288834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.288829    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-http\") pod \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") "
Oct 30 18:52:39.289137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.288855    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-grpc\") pod \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") "
Oct 30 18:52:39.289137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.288930    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config\") pod \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") "
Oct 30 18:52:39.289137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.288989    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-test-loki-lifecycle-secret\") pod \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\" (UID: \"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5\") "
Oct 30 18:52:39.289282 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.289187    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-ca-bundle" pod "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5"). InnerVolumeSpecName "data-science-usage-logs-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:39.289282 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.289224    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.289379 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.289327    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:39.289443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.289373    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.289779 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.289750    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config" pod "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:39.291243 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.291214    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-kube-api-access-vjkmw" pod "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5"). InnerVolumeSpecName "kube-api-access-vjkmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:39.291351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.291246    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-grpc" pod "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5"). InnerVolumeSpecName "data-science-usage-logs-lokistack-querier-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:39.291351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.291265    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-test-loki-lifecycle-secret" pod "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:39.291351 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.291286    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-http" pod "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" (UID: "fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5"). InnerVolumeSpecName "data-science-usage-logs-lokistack-querier-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:39.303220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.303198    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") DevicePath \"csi-63fc16ee3224580d120b91f6212f722820785db16c0cdcdd88fdc3f8519dd2ad\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.390422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.390392    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjkmw\" (UniqueName: \"kubernetes.io/projected/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-kube-api-access-vjkmw\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:39.390422 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.390420    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-querier-http\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-http\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:39.390590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.390431    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-querier-grpc\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-data-science-usage-logs-lokistack-querier-grpc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:39.390590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.390444    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:39.390590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.390453    5746 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5-test-loki-lifecycle-secret\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:52:39.406314 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.406288    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/49ae33b5ff4014339351ebe89a3549f1eae97ccd8a0289bad6d6f86f76fdff92/globalmount\"" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.434890 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.434866    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"data-science-usage-logs-lokistack-index-gateway-0\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") " pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:39.547383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.547304    5746 generic.go:356] "Generic (PLEG): container finished" podID="fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" containerID="2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a" exitCode=1
Oct 30 18:52:39.547383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.547349    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" event={"ID":"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5","Type":"ContainerDied","Data":"2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a"}
Oct 30 18:52:39.547383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.547354    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"
Oct 30 18:52:39.547659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.547387    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4" event={"ID":"fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5","Type":"ContainerDied","Data":"696d0daadfa1e823b2ea607d6492aaeb03e9013d2fc7074fe92bbe071140b585"}
Oct 30 18:52:39.547659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.547405    5746 scope.go:122] "RemoveContainer" containerID="2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a"
Oct 30 18:52:39.557027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.556962    5746 scope.go:122] "RemoveContainer" containerID="2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a"
Oct 30 18:52:39.557311 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:39.557285    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a\": container with ID starting with 2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a not found: ID does not exist" containerID="2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a"
Oct 30 18:52:39.557393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.557323    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a"} err="failed to get container status \"2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a\": rpc error: code = NotFound desc = could not find container \"2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a\": container with ID starting with 2435b6d4544e649259db359fea2ac0bdb7f9cb1a68c27b81f2d08dbd6eac8b1a not found: ID does not exist"
Oct 30 18:52:39.599521 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.599495    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"]
Oct 30 18:52:39.605574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.605551    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-querier-c94bb499b-t5mp4"]
Oct 30 18:52:39.690934 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:39.690884    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:40.000862 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:40.000837    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:52:40.004533 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:52:40.004505    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1234b4ef_79ca_43db_98cf_00dd8c75fb85.slice/crio-bb5e56766e2bc13ebc89db40be060ba962dac88a67cf592351daf1e152c0e553 WatchSource:0}: Error finding container bb5e56766e2bc13ebc89db40be060ba962dac88a67cf592351daf1e152c0e553: Status 404 returned error can't find the container with id bb5e56766e2bc13ebc89db40be060ba962dac88a67cf592351daf1e152c0e553
Oct 30 18:52:40.009218 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:40.009191    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" path="/var/lib/kubelet/pods/fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5/volumes"
Oct 30 18:52:40.142025 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:40.141985    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:52:40.554771 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:40.554735    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" event={"ID":"1234b4ef-79ca-43db-98cf-00dd8c75fb85","Type":"ContainerStarted","Data":"25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02"}
Oct 30 18:52:40.554962 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:40.554777    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" event={"ID":"1234b4ef-79ca-43db-98cf-00dd8c75fb85","Type":"ContainerStarted","Data":"bb5e56766e2bc13ebc89db40be060ba962dac88a67cf592351daf1e152c0e553"}
Oct 30 18:52:40.554962 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:40.554830    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:52:40.596369 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:40.596319    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" podStartSLOduration=2.5963046419999998 podStartE2EDuration="2.596304642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:40.596300953 +0000 UTC m=+3330.763119091" watchObservedRunningTime="2026-10-30 18:52:40.596304642 +0000 UTC m=+3330.763122769"
Oct 30 18:52:42.790045 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:52:42.790011    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:52:50.140066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:50.140025    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:52:50.140465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:52:50.140120    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0"
Oct 30 18:53:00.140441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:00.140394    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:53:01.565112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:01.565084    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:53:10.140216 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:10.140181    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:53:12.790748 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:53:12.790714    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:53:20.140846 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:20.140800    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:53:30.140062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:30.140017    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:53:40.140849 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:40.140809    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:53:42.790297 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:53:42.790260    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:53:43.138171 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.138132    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"]
Oct 30 18:53:43.138458 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.138428    5746 cpu_manager.go:414] "containerMap: removing container" podUID="fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" containerName="loki-querier"
Oct 30 18:53:43.138458 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.138444    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" containerName="loki-querier"
Oct 30 18:53:43.138608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.138527    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdc8ea47-213f-4afa-8506-7b6a9cd7a2d5" containerName="loki-querier"
Oct 30 18:53:43.141308 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.141293    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.163072 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.163044    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-headless-tls\""
Oct 30 18:53:43.171336 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.171304    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"]
Oct 30 18:53:43.171487 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.171449    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-9d8884b6\""
Oct 30 18:53:43.171563 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.171530    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-usage-logs-collector-dockercfg-rqrn7\""
Oct 30 18:53:43.214196 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.214171    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dqmh\" (UniqueName: \"kubernetes.io/projected/fbf389d4-a2d9-4090-abce-d868db64dc32-kube-api-access-8dqmh\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.214340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.214229    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf389d4-a2d9-4090-abce-d868db64dc32-otlp-tls-certs\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.214393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.214352    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbf389d4-a2d9-4090-abce-d868db64dc32-otc-internal\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.314867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.314829    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbf389d4-a2d9-4090-abce-d868db64dc32-otc-internal\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.314867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.314866    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dqmh\" (UniqueName: \"kubernetes.io/projected/fbf389d4-a2d9-4090-abce-d868db64dc32-kube-api-access-8dqmh\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.315088 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.314942    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf389d4-a2d9-4090-abce-d868db64dc32-otlp-tls-certs\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.315492 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.315465    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbf389d4-a2d9-4090-abce-d868db64dc32-otc-internal\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.317381 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.317359    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf389d4-a2d9-4090-abce-d868db64dc32-otlp-tls-certs\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.359941 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.359899    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dqmh\" (UniqueName: \"kubernetes.io/projected/fbf389d4-a2d9-4090-abce-d868db64dc32-kube-api-access-8dqmh\") pod \"data-science-usage-logs-collector-74fd5665b8-zptsh\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") " pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.454166 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.454073    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:43.753957 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.753859    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"]
Oct 30 18:53:43.757662 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:53:43.757634    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf389d4_a2d9_4090_abce_d868db64dc32.slice/crio-b4ec1c4bd502f90542aa1a22782da792c3d2b3b7ab0e4114cd5cf8d8776d7822 WatchSource:0}: Error finding container b4ec1c4bd502f90542aa1a22782da792c3d2b3b7ab0e4114cd5cf8d8776d7822: Status 404 returned error can't find the container with id b4ec1c4bd502f90542aa1a22782da792c3d2b3b7ab0e4114cd5cf8d8776d7822
Oct 30 18:53:43.760109 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.760075    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:53:43.760207 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.760133    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:53:43.814308 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.814279    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh" event={"ID":"fbf389d4-a2d9-4090-abce-d868db64dc32","Type":"ContainerStarted","Data":"a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033"}
Oct 30 18:53:43.814716 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.814350    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh" event={"ID":"fbf389d4-a2d9-4090-abce-d868db64dc32","Type":"ContainerStarted","Data":"b4ec1c4bd502f90542aa1a22782da792c3d2b3b7ab0e4114cd5cf8d8776d7822"}
Oct 30 18:53:43.859283 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:43.859230    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh" podStartSLOduration=0.859216526 podStartE2EDuration="859.216526ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:43.85850977 +0000 UTC m=+3394.025327896" watchObservedRunningTime="2026-10-30 18:53:43.859216526 +0000 UTC m=+3394.026034715"
Oct 30 18:53:46.791002 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:46.790967    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:53:46.791402 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:46.791171    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" podUID="1234b4ef-79ca-43db-98cf-00dd8c75fb85" containerName="loki-index-gateway" containerID="cri-o://25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02" gracePeriod=30
Oct 30 18:53:46.904509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:46.903344    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"]
Oct 30 18:53:46.904509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:46.903554    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh" podUID="fbf389d4-a2d9-4090-abce-d868db64dc32" containerName="otc-container" containerID="cri-o://a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033" gracePeriod=30
Oct 30 18:53:47.054285 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.054230    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:53:47.151264 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151241    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-config\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-config\") pod \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") "
Oct 30 18:53:47.151426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151298    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1234b4ef-79ca-43db-98cf-00dd8c75fb85-kube-api-access-dxvlh\" (UniqueName: \"kubernetes.io/projected/1234b4ef-79ca-43db-98cf-00dd8c75fb85-kube-api-access-dxvlh\") pod \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") "
Oct 30 18:53:47.151426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151330    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-grpc\") pod \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") "
Oct 30 18:53:47.151426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151388    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-test-loki-lifecycle-secret\") pod \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") "
Oct 30 18:53:47.151608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151482    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") pod \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") "
Oct 30 18:53:47.151608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151518    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-ca-bundle\") pod \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") "
Oct 30 18:53:47.151608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151547    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-http\") pod \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\" (UID: \"1234b4ef-79ca-43db-98cf-00dd8c75fb85\") "
Oct 30 18:53:47.151759 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151647    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-config" pod "1234b4ef-79ca-43db-98cf-00dd8c75fb85" (UID: "1234b4ef-79ca-43db-98cf-00dd8c75fb85"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:53:47.151841 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.151822    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.152685 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.152426    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-ca-bundle" pod "1234b4ef-79ca-43db-98cf-00dd8c75fb85" (UID: "1234b4ef-79ca-43db-98cf-00dd8c75fb85"). InnerVolumeSpecName "data-science-usage-logs-lokistack-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:53:47.153940 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.153896    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-grpc" pod "1234b4ef-79ca-43db-98cf-00dd8c75fb85" (UID: "1234b4ef-79ca-43db-98cf-00dd8c75fb85"). InnerVolumeSpecName "data-science-usage-logs-lokistack-index-gateway-grpc". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:53:47.154302 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.154275    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1234b4ef-79ca-43db-98cf-00dd8c75fb85-kube-api-access-dxvlh" pod "1234b4ef-79ca-43db-98cf-00dd8c75fb85" (UID: "1234b4ef-79ca-43db-98cf-00dd8c75fb85"). InnerVolumeSpecName "kube-api-access-dxvlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:53:47.154389 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.154315    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-test-loki-lifecycle-secret" pod "1234b4ef-79ca-43db-98cf-00dd8c75fb85" (UID: "1234b4ef-79ca-43db-98cf-00dd8c75fb85"). InnerVolumeSpecName "test-loki-lifecycle-secret". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:53:47.154534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.154517    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-http" pod "1234b4ef-79ca-43db-98cf-00dd8c75fb85" (UID: "1234b4ef-79ca-43db-98cf-00dd8c75fb85"). InnerVolumeSpecName "data-science-usage-logs-lokistack-index-gateway-http". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:53:47.155134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.155116    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3" pod "1234b4ef-79ca-43db-98cf-00dd8c75fb85" (UID: "1234b4ef-79ca-43db-98cf-00dd8c75fb85"). InnerVolumeSpecName "pvc-40bb1491-40e8-4bff-a340-7475b8c784f4". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:53:47.166112 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.166094    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:47.252965 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.252931    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbf389d4-a2d9-4090-abce-d868db64dc32-kube-api-access-8dqmh\" (UniqueName: \"kubernetes.io/projected/fbf389d4-a2d9-4090-abce-d868db64dc32-kube-api-access-8dqmh\") pod \"fbf389d4-a2d9-4090-abce-d868db64dc32\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") "
Oct 30 18:53:47.253147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253022    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbf389d4-a2d9-4090-abce-d868db64dc32-otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf389d4-a2d9-4090-abce-d868db64dc32-otlp-tls-certs\") pod \"fbf389d4-a2d9-4090-abce-d868db64dc32\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") "
Oct 30 18:53:47.253147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253049    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbf389d4-a2d9-4090-abce-d868db64dc32-otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbf389d4-a2d9-4090-abce-d868db64dc32-otc-internal\") pod \"fbf389d4-a2d9-4090-abce-d868db64dc32\" (UID: \"fbf389d4-a2d9-4090-abce-d868db64dc32\") "
Oct 30 18:53:47.253288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253203    5746 reconciler_common.go:299] "Volume detached for volume \"test-loki-lifecycle-secret\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-test-loki-lifecycle-secret\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.253288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253232    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:53:47.253288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253247    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-ca-bundle\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.253288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253257    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-index-gateway-http\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-http\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.253288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253271    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxvlh\" (UniqueName: \"kubernetes.io/projected/1234b4ef-79ca-43db-98cf-00dd8c75fb85-kube-api-access-dxvlh\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.253288 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253286    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-usage-logs-lokistack-index-gateway-grpc\" (UniqueName: \"kubernetes.io/secret/1234b4ef-79ca-43db-98cf-00dd8c75fb85-data-science-usage-logs-lokistack-index-gateway-grpc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.253607 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.253477    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf389d4-a2d9-4090-abce-d868db64dc32-otc-internal" pod "fbf389d4-a2d9-4090-abce-d868db64dc32" (UID: "fbf389d4-a2d9-4090-abce-d868db64dc32"). InnerVolumeSpecName "otc-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:53:47.255263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.255237    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf389d4-a2d9-4090-abce-d868db64dc32-kube-api-access-8dqmh" pod "fbf389d4-a2d9-4090-abce-d868db64dc32" (UID: "fbf389d4-a2d9-4090-abce-d868db64dc32"). InnerVolumeSpecName "kube-api-access-8dqmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:53:47.255431 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.255411    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbf389d4-a2d9-4090-abce-d868db64dc32-otlp-tls-certs" pod "fbf389d4-a2d9-4090-abce-d868db64dc32" (UID: "fbf389d4-a2d9-4090-abce-d868db64dc32"). InnerVolumeSpecName "otlp-tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:53:47.262210 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.262193    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-40bb1491-40e8-4bff-a340-7475b8c784f4" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:53:47.354357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.354286    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-40bb1491-40e8-4bff-a340-7475b8c784f4\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03360a151e08208c3\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-63fc16ee3224580d120b91f6212f722820785db16c0cdcdd88fdc3f8519dd2ad\""
Oct 30 18:53:47.354357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.354312    5746 reconciler_common.go:299] "Volume detached for volume \"otlp-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf389d4-a2d9-4090-abce-d868db64dc32-otlp-tls-certs\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.354357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.354323    5746 reconciler_common.go:299] "Volume detached for volume \"otc-internal\" (UniqueName: \"kubernetes.io/configmap/fbf389d4-a2d9-4090-abce-d868db64dc32-otc-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.354357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.354331    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dqmh\" (UniqueName: \"kubernetes.io/projected/fbf389d4-a2d9-4090-abce-d868db64dc32-kube-api-access-8dqmh\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:53:47.832066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.832036    5746 generic.go:356] "Generic (PLEG): container finished" podID="fbf389d4-a2d9-4090-abce-d868db64dc32" containerID="a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033" exitCode=0
Oct 30 18:53:47.832509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.832084    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"
Oct 30 18:53:47.832509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.832120    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh" event={"ID":"fbf389d4-a2d9-4090-abce-d868db64dc32","Type":"ContainerDied","Data":"a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033"}
Oct 30 18:53:47.832509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.832180    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh" event={"ID":"fbf389d4-a2d9-4090-abce-d868db64dc32","Type":"ContainerDied","Data":"b4ec1c4bd502f90542aa1a22782da792c3d2b3b7ab0e4114cd5cf8d8776d7822"}
Oct 30 18:53:47.832509 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.832214    5746 scope.go:122] "RemoveContainer" containerID="a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033"
Oct 30 18:53:47.833621 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.833601    5746 generic.go:356] "Generic (PLEG): container finished" podID="1234b4ef-79ca-43db-98cf-00dd8c75fb85" containerID="25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02" exitCode=0
Oct 30 18:53:47.833696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.833638    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" event={"ID":"1234b4ef-79ca-43db-98cf-00dd8c75fb85","Type":"ContainerDied","Data":"25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02"}
Oct 30 18:53:47.833696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.833658    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"
Oct 30 18:53:47.833696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.833678    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0" event={"ID":"1234b4ef-79ca-43db-98cf-00dd8c75fb85","Type":"ContainerDied","Data":"bb5e56766e2bc13ebc89db40be060ba962dac88a67cf592351daf1e152c0e553"}
Oct 30 18:53:47.841133 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.841112    5746 scope.go:122] "RemoveContainer" containerID="a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033"
Oct 30 18:53:47.841470 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:53:47.841449    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033\": container with ID starting with a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033 not found: ID does not exist" containerID="a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033"
Oct 30 18:53:47.841528 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.841484    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033"} err="failed to get container status \"a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033\": rpc error: code = NotFound desc = could not find container \"a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033\": container with ID starting with a6758dd1ea5ed641f10b076ef0ca15abfd7c689d37bdcf3701aeb3c687008033 not found: ID does not exist"
Oct 30 18:53:47.841528 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.841502    5746 scope.go:122] "RemoveContainer" containerID="25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02"
Oct 30 18:53:47.849833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.849815    5746 scope.go:122] "RemoveContainer" containerID="25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02"
Oct 30 18:53:47.850120 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:53:47.850101    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02\": container with ID starting with 25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02 not found: ID does not exist" containerID="25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02"
Oct 30 18:53:47.850181 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.850131    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02"} err="failed to get container status \"25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02\": rpc error: code = NotFound desc = could not find container \"25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02\": container with ID starting with 25eb5745a0b8dae37af2eb660f8c38486b7b80b82f12cf8473bd0280a192cb02 not found: ID does not exist"
Oct 30 18:53:47.881393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.881366    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:53:47.893336 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.893312    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-lokistack-index-gateway-0"]
Oct 30 18:53:47.918456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.918429    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"]
Oct 30 18:53:47.922581 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.922558    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-usage-logs-collector-74fd5665b8-zptsh"]
Oct 30 18:53:47.966136 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.966110    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:53:47.966278 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:47.966257    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/seaweedfs" podUID="790dc600-ad91-4202-b94f-049ed796609e" containerName="seaweedfs" containerID="cri-o://59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0" gracePeriod=30
Oct 30 18:53:48.008614 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:48.008585    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1234b4ef-79ca-43db-98cf-00dd8c75fb85" path="/var/lib/kubelet/pods/1234b4ef-79ca-43db-98cf-00dd8c75fb85/volumes"
Oct 30 18:53:48.008994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:48.008980    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbf389d4-a2d9-4090-abce-d868db64dc32" path="/var/lib/kubelet/pods/fbf389d4-a2d9-4090-abce-d868db64dc32/volumes"
Oct 30 18:53:50.140617 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:53:50.140579    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:54:00.140079 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:00.140039    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:54:10.140194 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:10.140149    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:54:12.789190 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:54:12.789158    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:54:13.210734 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.210707    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:54:13.279769 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.279738    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/790dc600-ad91-4202-b94f-049ed796609e-data\" (UniqueName: \"kubernetes.io/empty-dir/790dc600-ad91-4202-b94f-049ed796609e-data\") pod \"790dc600-ad91-4202-b94f-049ed796609e\" (UID: \"790dc600-ad91-4202-b94f-049ed796609e\") "
Oct 30 18:54:13.279988 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.279793    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/790dc600-ad91-4202-b94f-049ed796609e-kube-api-access-2k8hg\" (UniqueName: \"kubernetes.io/projected/790dc600-ad91-4202-b94f-049ed796609e-kube-api-access-2k8hg\") pod \"790dc600-ad91-4202-b94f-049ed796609e\" (UID: \"790dc600-ad91-4202-b94f-049ed796609e\") "
Oct 30 18:54:13.281162 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.281129    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/790dc600-ad91-4202-b94f-049ed796609e-data" pod "790dc600-ad91-4202-b94f-049ed796609e" (UID: "790dc600-ad91-4202-b94f-049ed796609e"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:13.282225 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.282191    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/790dc600-ad91-4202-b94f-049ed796609e-kube-api-access-2k8hg" pod "790dc600-ad91-4202-b94f-049ed796609e" (UID: "790dc600-ad91-4202-b94f-049ed796609e"). InnerVolumeSpecName "kube-api-access-2k8hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:13.380360 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.380332    5746 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/790dc600-ad91-4202-b94f-049ed796609e-data\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:54:13.380360 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.380359    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k8hg\" (UniqueName: \"kubernetes.io/projected/790dc600-ad91-4202-b94f-049ed796609e-kube-api-access-2k8hg\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:54:13.937656 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.937621    5746 generic.go:356] "Generic (PLEG): container finished" podID="790dc600-ad91-4202-b94f-049ed796609e" containerID="59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0" exitCode=0
Oct 30 18:54:13.938098 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.937677    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/seaweedfs"
Oct 30 18:54:13.938098 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.937710    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/seaweedfs" event={"ID":"790dc600-ad91-4202-b94f-049ed796609e","Type":"ContainerDied","Data":"59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0"}
Oct 30 18:54:13.938098 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.937754    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/seaweedfs" event={"ID":"790dc600-ad91-4202-b94f-049ed796609e","Type":"ContainerDied","Data":"6262ab26f2a0adc289300f705931b68c105d122bb6a290f8201b5935ec4c9022"}
Oct 30 18:54:13.938098 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.937771    5746 scope.go:122] "RemoveContainer" containerID="59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0"
Oct 30 18:54:13.946450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.946437    5746 scope.go:122] "RemoveContainer" containerID="59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0"
Oct 30 18:54:13.946715 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:54:13.946696    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0\": container with ID starting with 59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0 not found: ID does not exist" containerID="59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0"
Oct 30 18:54:13.946783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.946728    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0"} err="failed to get container status \"59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0\": rpc error: code = NotFound desc = could not find container \"59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0\": container with ID starting with 59ef3f3d0f7c077e14b12ac40e8f2939193993d66e4be506d220ba34598d34c0 not found: ID does not exist"
Oct 30 18:54:13.988137 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.988116    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:54:13.992153 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:13.992135    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/seaweedfs"]
Oct 30 18:54:14.007825 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:14.007797    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="790dc600-ad91-4202-b94f-049ed796609e" path="/var/lib/kubelet/pods/790dc600-ad91-4202-b94f-049ed796609e/volumes"
Oct 30 18:54:20.140471 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:20.140389    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:54:22.192220 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192182    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"]
Oct 30 18:54:22.192601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192582    5746 cpu_manager.go:414] "containerMap: removing container" podUID="1234b4ef-79ca-43db-98cf-00dd8c75fb85" containerName="loki-index-gateway"
Oct 30 18:54:22.192601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192598    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1234b4ef-79ca-43db-98cf-00dd8c75fb85" containerName="loki-index-gateway"
Oct 30 18:54:22.192677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192629    5746 cpu_manager.go:414] "containerMap: removing container" podUID="790dc600-ad91-4202-b94f-049ed796609e" containerName="seaweedfs"
Oct 30 18:54:22.192677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192637    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="790dc600-ad91-4202-b94f-049ed796609e" containerName="seaweedfs"
Oct 30 18:54:22.192739 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192699    5746 cpu_manager.go:414] "containerMap: removing container" podUID="fbf389d4-a2d9-4090-abce-d868db64dc32" containerName="otc-container"
Oct 30 18:54:22.192739 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192709    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbf389d4-a2d9-4090-abce-d868db64dc32" containerName="otc-container"
Oct 30 18:54:22.192842 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192777    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="790dc600-ad91-4202-b94f-049ed796609e" containerName="seaweedfs"
Oct 30 18:54:22.192842 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192790    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbf389d4-a2d9-4090-abce-d868db64dc32" containerName="otc-container"
Oct 30 18:54:22.192842 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.192800    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="1234b4ef-79ca-43db-98cf-00dd8c75fb85" containerName="loki-index-gateway"
Oct 30 18:54:22.197226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.197203    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"
Oct 30 18:54:22.213257 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.213235    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"]
Oct 30 18:54:22.224258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.224239    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:54:22.235562 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.235539    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq2sf\" (UniqueName: \"kubernetes.io/projected/44338d20-f1cb-40c7-acdc-ba321f4b35ae-kube-api-access-wq2sf\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t\" (UID: \"44338d20-f1cb-40c7-acdc-ba321f4b35ae\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"
Oct 30 18:54:22.336436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.336407    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2sf\" (UniqueName: \"kubernetes.io/projected/44338d20-f1cb-40c7-acdc-ba321f4b35ae-kube-api-access-wq2sf\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t\" (UID: \"44338d20-f1cb-40c7-acdc-ba321f4b35ae\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"
Oct 30 18:54:22.360257 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.360227    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq2sf\" (UniqueName: \"kubernetes.io/projected/44338d20-f1cb-40c7-acdc-ba321f4b35ae-kube-api-access-wq2sf\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t\" (UID: \"44338d20-f1cb-40c7-acdc-ba321f4b35ae\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"
Oct 30 18:54:22.508857 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.508772    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"
Oct 30 18:54:22.801449 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.801377    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"]
Oct 30 18:54:22.805344 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:54:22.805316    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44338d20_f1cb_40c7_acdc_ba321f4b35ae.slice/crio-d42c78727d300c6e671ed2e56dadc94d24875f227371f307fc5e17d66ccd5464 WatchSource:0}: Error finding container d42c78727d300c6e671ed2e56dadc94d24875f227371f307fc5e17d66ccd5464: Status 404 returned error can't find the container with id d42c78727d300c6e671ed2e56dadc94d24875f227371f307fc5e17d66ccd5464
Oct 30 18:54:22.975282 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.975248    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t" event={"ID":"44338d20-f1cb-40c7-acdc-ba321f4b35ae","Type":"ContainerStarted","Data":"5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791"}
Oct 30 18:54:22.975458 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:22.975291    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t" event={"ID":"44338d20-f1cb-40c7-acdc-ba321f4b35ae","Type":"ContainerStarted","Data":"d42c78727d300c6e671ed2e56dadc94d24875f227371f307fc5e17d66ccd5464"}
Oct 30 18:54:23.013662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.013612    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t" podStartSLOduration=1.013593433 podStartE2EDuration="1.013593433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:23.013023185 +0000 UTC m=+3433.179841313" watchObservedRunningTime="2026-10-30 18:54:23.013593433 +0000 UTC m=+3433.180411560"
Oct 30 18:54:23.206438 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.206406    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:54:23.209561 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.209540    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.229336 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.229315    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:54:23.283297 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.283269    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:54:23.295711 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.295685    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"]
Oct 30 18:54:23.298712 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.298695    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.321583 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.321053    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:54:23.321583 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.321189    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-9bsg9\""
Oct 30 18:54:23.322149 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.322123    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"]
Oct 30 18:54:23.344749 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.344725    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/44e6260e-6dd0-4b5d-a028-61066f52dd7e-plugins\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.344852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.344779    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mph2\" (UniqueName: \"kubernetes.io/projected/44e6260e-6dd0-4b5d-a028-61066f52dd7e-kube-api-access-7mph2\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.344852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.344806    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44e6260e-6dd0-4b5d-a028-61066f52dd7e-config\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.360308 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.360285    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:54:23.445771 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.445746    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44e6260e-6dd0-4b5d-a028-61066f52dd7e-config\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.445865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.445791    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/236d6a4c-fcca-4427-9495-74bc7027727e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.445865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.445814    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/44e6260e-6dd0-4b5d-a028-61066f52dd7e-plugins\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.445865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.445835    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4v6s\" (UniqueName: \"kubernetes.io/projected/236d6a4c-fcca-4427-9495-74bc7027727e-kube-api-access-f4v6s\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.446010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.445931    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/236d6a4c-fcca-4427-9495-74bc7027727e-ta-internal\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.446048 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.446021    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mph2\" (UniqueName: \"kubernetes.io/projected/44e6260e-6dd0-4b5d-a028-61066f52dd7e-kube-api-access-7mph2\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.446333 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.446315    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/44e6260e-6dd0-4b5d-a028-61066f52dd7e-plugins\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.446471 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.446453    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44e6260e-6dd0-4b5d-a028-61066f52dd7e-config\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.471429 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.471385    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mph2\" (UniqueName: \"kubernetes.io/projected/44e6260e-6dd0-4b5d-a028-61066f52dd7e-kube-api-access-7mph2\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:23.547349 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.547322    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/236d6a4c-fcca-4427-9495-74bc7027727e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.547472 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.547362    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4v6s\" (UniqueName: \"kubernetes.io/projected/236d6a4c-fcca-4427-9495-74bc7027727e-kube-api-access-f4v6s\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.547472 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.547382    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/236d6a4c-fcca-4427-9495-74bc7027727e-ta-internal\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.547992 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.547975    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/236d6a4c-fcca-4427-9495-74bc7027727e-ta-internal\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.549889 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.549871    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/236d6a4c-fcca-4427-9495-74bc7027727e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.570536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.570510    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4v6s\" (UniqueName: \"kubernetes.io/projected/236d6a4c-fcca-4427-9495-74bc7027727e-kube-api-access-f4v6s\") pod \"data-science-collector-targetallocator-5cc94cbbf9-2c5sb\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.609662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.609639    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.904590 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.904556    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"]
Oct 30 18:54:23.907851 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:54:23.907823    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod236d6a4c_fcca_4427_9495_74bc7027727e.slice/crio-4c6705430833cb6d0f84584d6ccf35b15daae963fcd283ba12c71a4bd3f03a12 WatchSource:0}: Error finding container 4c6705430833cb6d0f84584d6ccf35b15daae963fcd283ba12c71a4bd3f03a12: Status 404 returned error can't find the container with id 4c6705430833cb6d0f84584d6ccf35b15daae963fcd283ba12c71a4bd3f03a12
Oct 30 18:54:23.910148 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.910109    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:54:23.910229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.910171    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:54:23.981810 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.981780    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb" event={"ID":"236d6a4c-fcca-4427-9495-74bc7027727e","Type":"ContainerStarted","Data":"07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0"}
Oct 30 18:54:23.981978 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.981823    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb" event={"ID":"236d6a4c-fcca-4427-9495-74bc7027727e","Type":"ContainerStarted","Data":"4c6705430833cb6d0f84584d6ccf35b15daae963fcd283ba12c71a4bd3f03a12"}
Oct 30 18:54:23.982058 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.982048    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:23.983470 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:23.983444    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb" podUID="236d6a4c-fcca-4427-9495-74bc7027727e" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.158:8080/readyz\": dial tcp 10.133.0.158:8080: connect: connection refused"
Oct 30 18:54:24.022534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:24.022491    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb" podStartSLOduration=1.022479524 podStartE2EDuration="1.022479524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:24.022271912 +0000 UTC m=+3434.189090039" watchObservedRunningTime="2026-10-30 18:54:24.022479524 +0000 UTC m=+3434.189297645"
Oct 30 18:54:24.986791 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:24.986764    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:54:30.140233 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:30.140196    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:54:30.991800 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:30.991770    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"]
Oct 30 18:54:30.991992 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:30.991946    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t" podUID="44338d20-f1cb-40c7-acdc-ba321f4b35ae" containerName="thanos-querier" containerID="cri-o://5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791" gracePeriod=30
Oct 30 18:54:31.230535 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.230510    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"
Oct 30 18:54:31.413565 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.413534    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44338d20-f1cb-40c7-acdc-ba321f4b35ae-kube-api-access-wq2sf\" (UniqueName: \"kubernetes.io/projected/44338d20-f1cb-40c7-acdc-ba321f4b35ae-kube-api-access-wq2sf\") pod \"44338d20-f1cb-40c7-acdc-ba321f4b35ae\" (UID: \"44338d20-f1cb-40c7-acdc-ba321f4b35ae\") "
Oct 30 18:54:31.415809 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.415784    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44338d20-f1cb-40c7-acdc-ba321f4b35ae-kube-api-access-wq2sf" pod "44338d20-f1cb-40c7-acdc-ba321f4b35ae" (UID: "44338d20-f1cb-40c7-acdc-ba321f4b35ae"). InnerVolumeSpecName "kube-api-access-wq2sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:31.514821 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.514794    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:31.515047 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.514859    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq2sf\" (UniqueName: \"kubernetes.io/projected/44338d20-f1cb-40c7-acdc-ba321f4b35ae-kube-api-access-wq2sf\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:54:31.542889 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.542862    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:31.615864 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.615834    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:31.616050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.615977    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:31.629853 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.629823    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:31.730004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.729976    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:31.777001 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.776976    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:31.920589 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:31.920560    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:32.016676 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.016651    5746 generic.go:356] "Generic (PLEG): container finished" podID="44338d20-f1cb-40c7-acdc-ba321f4b35ae" containerID="5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791" exitCode=0
Oct 30 18:54:32.016776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.016697    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"
Oct 30 18:54:32.016776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.016720    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t" event={"ID":"44338d20-f1cb-40c7-acdc-ba321f4b35ae","Type":"ContainerDied","Data":"5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791"}
Oct 30 18:54:32.016776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.016756    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t" event={"ID":"44338d20-f1cb-40c7-acdc-ba321f4b35ae","Type":"ContainerDied","Data":"d42c78727d300c6e671ed2e56dadc94d24875f227371f307fc5e17d66ccd5464"}
Oct 30 18:54:32.016776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.016774    5746 scope.go:122] "RemoveContainer" containerID="5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791"
Oct 30 18:54:32.024507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.024491    5746 scope.go:122] "RemoveContainer" containerID="5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791"
Oct 30 18:54:32.024770 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:54:32.024749    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791\": container with ID starting with 5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791 not found: ID does not exist" containerID="5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791"
Oct 30 18:54:32.024868 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.024775    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791"} err="failed to get container status \"5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791\": rpc error: code = NotFound desc = could not find container \"5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791\": container with ID starting with 5986f5f0a67e3773d4d126a5f781dac649a35cd71b7b2ff5e7a0728b8d541791 not found: ID does not exist"
Oct 30 18:54:32.064960 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.064934    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"]
Oct 30 18:54:32.070264 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.070240    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-cnd6t"]
Oct 30 18:54:32.213052 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:32.213015    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:54:32.217088 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:54:32.217062    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44e6260e_6dd0_4b5d_a028_61066f52dd7e.slice/crio-03e3e08172349ee5e9a89da50e304f67a25ce8ff36ec6d8f8362b1d980b0455b WatchSource:0}: Error finding container 03e3e08172349ee5e9a89da50e304f67a25ce8ff36ec6d8f8362b1d980b0455b: Status 404 returned error can't find the container with id 03e3e08172349ee5e9a89da50e304f67a25ce8ff36ec6d8f8362b1d980b0455b
Oct 30 18:54:33.023342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:33.023309    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"44e6260e-6dd0-4b5d-a028-61066f52dd7e","Type":"ContainerStarted","Data":"8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f"}
Oct 30 18:54:33.023819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:33.023352    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"44e6260e-6dd0-4b5d-a028-61066f52dd7e","Type":"ContainerStarted","Data":"03e3e08172349ee5e9a89da50e304f67a25ce8ff36ec6d8f8362b1d980b0455b"}
Oct 30 18:54:33.023819 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:33.023403    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:33.064122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:33.064072    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=10.064057135 podStartE2EDuration="10.064057135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:33.063122694 +0000 UTC m=+3443.229940832" watchObservedRunningTime="2026-10-30 18:54:33.064057135 +0000 UTC m=+3443.230875262"
Oct 30 18:54:34.009258 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:34.009223    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44338d20-f1cb-40c7-acdc-ba321f4b35ae" path="/var/lib/kubelet/pods/44338d20-f1cb-40c7-acdc-ba321f4b35ae/volumes"
Oct 30 18:54:40.139900 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:40.139863    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:54:42.790311 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:54:42.790267    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:54:44.032172 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:44.032149    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:54:50.140249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:54:50.140206    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:55:00.140787 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:00.140751    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:55:08.094160 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.094125    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"]
Oct 30 18:55:08.094641 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.094391    5746 cpu_manager.go:414] "containerMap: removing container" podUID="44338d20-f1cb-40c7-acdc-ba321f4b35ae" containerName="thanos-querier"
Oct 30 18:55:08.094641 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.094402    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44338d20-f1cb-40c7-acdc-ba321f4b35ae" containerName="thanos-querier"
Oct 30 18:55:08.094641 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.094473    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="44338d20-f1cb-40c7-acdc-ba321f4b35ae" containerName="thanos-querier"
Oct 30 18:55:08.109405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.109379    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.116728 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.116704    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"]
Oct 30 18:55:08.231984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.231941    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plm6x\" (UniqueName: \"kubernetes.io/projected/96789eae-245f-48eb-81af-7a4f8c7bd831-kube-api-access-plm6x\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.232173 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.231999    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/96789eae-245f-48eb-81af-7a4f8c7bd831-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.232173 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.232090    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/96789eae-245f-48eb-81af-7a4f8c7bd831-ta-internal\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.333439 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.333403    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plm6x\" (UniqueName: \"kubernetes.io/projected/96789eae-245f-48eb-81af-7a4f8c7bd831-kube-api-access-plm6x\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.333439 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.333445    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/96789eae-245f-48eb-81af-7a4f8c7bd831-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.333673 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.333483    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/96789eae-245f-48eb-81af-7a4f8c7bd831-ta-internal\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.334091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.334070    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/96789eae-245f-48eb-81af-7a4f8c7bd831-ta-internal\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.336120 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.336098    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/96789eae-245f-48eb-81af-7a4f8c7bd831-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.356323 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.356294    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plm6x\" (UniqueName: \"kubernetes.io/projected/96789eae-245f-48eb-81af-7a4f8c7bd831-kube-api-access-plm6x\") pod \"data-science-collector-targetallocator-64df9b87d9-thv59\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.420974 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.420951    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:08.717261 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.717187    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"]
Oct 30 18:55:08.720522 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:55:08.720491    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96789eae_245f_48eb_81af_7a4f8c7bd831.slice/crio-e6608cab457757399e93688887a339d4b76d90a198118a6fe47d800e6c34eafc WatchSource:0}: Error finding container e6608cab457757399e93688887a339d4b76d90a198118a6fe47d800e6c34eafc: Status 404 returned error can't find the container with id e6608cab457757399e93688887a339d4b76d90a198118a6fe47d800e6c34eafc
Oct 30 18:55:08.722737 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.722701    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:55:08.722848 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:08.722777    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:55:09.165300 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.165264    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59" event={"ID":"96789eae-245f-48eb-81af-7a4f8c7bd831","Type":"ContainerStarted","Data":"499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5"}
Oct 30 18:55:09.165768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.165312    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59" event={"ID":"96789eae-245f-48eb-81af-7a4f8c7bd831","Type":"ContainerStarted","Data":"e6608cab457757399e93688887a339d4b76d90a198118a6fe47d800e6c34eafc"}
Oct 30 18:55:09.165768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.165475    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:09.166826 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.166808    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:09.206541 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.206454    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59" podStartSLOduration=1.206437834 podStartE2EDuration="1.206437834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:09.205883154 +0000 UTC m=+3479.372701291" watchObservedRunningTime="2026-10-30 18:55:09.206437834 +0000 UTC m=+3479.373255960"
Oct 30 18:55:09.286781 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.286744    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"]
Oct 30 18:55:09.287021 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.286947    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb" podUID="236d6a4c-fcca-4427-9495-74bc7027727e" containerName="ta-container" containerID="cri-o://07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0" gracePeriod=30
Oct 30 18:55:09.542584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.542560    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:55:09.643902 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.643874    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/236d6a4c-fcca-4427-9495-74bc7027727e-kube-api-access-f4v6s\" (UniqueName: \"kubernetes.io/projected/236d6a4c-fcca-4427-9495-74bc7027727e-kube-api-access-f4v6s\") pod \"236d6a4c-fcca-4427-9495-74bc7027727e\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") "
Oct 30 18:55:09.644123 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.644032    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/236d6a4c-fcca-4427-9495-74bc7027727e-ta-internal\" (UniqueName: \"kubernetes.io/configmap/236d6a4c-fcca-4427-9495-74bc7027727e-ta-internal\") pod \"236d6a4c-fcca-4427-9495-74bc7027727e\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") "
Oct 30 18:55:09.644123 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.644069    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/236d6a4c-fcca-4427-9495-74bc7027727e-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/236d6a4c-fcca-4427-9495-74bc7027727e-data-science-collector-ta-server-cert\") pod \"236d6a4c-fcca-4427-9495-74bc7027727e\" (UID: \"236d6a4c-fcca-4427-9495-74bc7027727e\") "
Oct 30 18:55:09.644489 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.644458    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/236d6a4c-fcca-4427-9495-74bc7027727e-ta-internal" pod "236d6a4c-fcca-4427-9495-74bc7027727e" (UID: "236d6a4c-fcca-4427-9495-74bc7027727e"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:55:09.646226 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.646195    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/236d6a4c-fcca-4427-9495-74bc7027727e-kube-api-access-f4v6s" pod "236d6a4c-fcca-4427-9495-74bc7027727e" (UID: "236d6a4c-fcca-4427-9495-74bc7027727e"). InnerVolumeSpecName "kube-api-access-f4v6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:09.646307 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.646235    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/236d6a4c-fcca-4427-9495-74bc7027727e-data-science-collector-ta-server-cert" pod "236d6a4c-fcca-4427-9495-74bc7027727e" (UID: "236d6a4c-fcca-4427-9495-74bc7027727e"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:55:09.744838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.744749    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/236d6a4c-fcca-4427-9495-74bc7027727e-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:55:09.744838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.744780    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/236d6a4c-fcca-4427-9495-74bc7027727e-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:55:09.744838 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:09.744796    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4v6s\" (UniqueName: \"kubernetes.io/projected/236d6a4c-fcca-4427-9495-74bc7027727e-kube-api-access-f4v6s\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:55:10.140708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.140670    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:55:10.171538 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.171502    5746 generic.go:356] "Generic (PLEG): container finished" podID="236d6a4c-fcca-4427-9495-74bc7027727e" containerID="07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0" exitCode=0
Oct 30 18:55:10.172101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.171573    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"
Oct 30 18:55:10.172101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.171599    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb" event={"ID":"236d6a4c-fcca-4427-9495-74bc7027727e","Type":"ContainerDied","Data":"07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0"}
Oct 30 18:55:10.172101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.171651    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb" event={"ID":"236d6a4c-fcca-4427-9495-74bc7027727e","Type":"ContainerDied","Data":"4c6705430833cb6d0f84584d6ccf35b15daae963fcd283ba12c71a4bd3f03a12"}
Oct 30 18:55:10.172101 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.171676    5746 scope.go:122] "RemoveContainer" containerID="07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0"
Oct 30 18:55:10.180394 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.180377    5746 scope.go:122] "RemoveContainer" containerID="07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0"
Oct 30 18:55:10.180677 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:55:10.180656    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0\": container with ID starting with 07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0 not found: ID does not exist" containerID="07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0"
Oct 30 18:55:10.180740 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.180689    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0"} err="failed to get container status \"07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0\": rpc error: code = NotFound desc = could not find container \"07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0\": container with ID starting with 07d2bf1b931e875675b629444e9e50f4c87ce54508e247e76f97dedd17a5c2f0 not found: ID does not exist"
Oct 30 18:55:10.221764 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.221741    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"]
Oct 30 18:55:10.226335 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:10.226314    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5cc94cbbf9-2c5sb"]
Oct 30 18:55:12.009239 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:12.009204    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="236d6a4c-fcca-4427-9495-74bc7027727e" path="/var/lib/kubelet/pods/236d6a4c-fcca-4427-9495-74bc7027727e/volumes"
Oct 30 18:55:12.789827 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:55:12.789792    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:55:20.139858 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:20.139814    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:55:25.550501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.550464    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"]
Oct 30 18:55:25.550921 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.550888    5746 cpu_manager.go:414] "containerMap: removing container" podUID="236d6a4c-fcca-4427-9495-74bc7027727e" containerName="ta-container"
Oct 30 18:55:25.550969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.550920    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="236d6a4c-fcca-4427-9495-74bc7027727e" containerName="ta-container"
Oct 30 18:55:25.551004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.550987    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="236d6a4c-fcca-4427-9495-74bc7027727e" containerName="ta-container"
Oct 30 18:55:25.555024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.555003    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.582397 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.582366    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"]
Oct 30 18:55:25.686088 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.686054    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.686291 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.686107    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckdr5\" (UniqueName: \"kubernetes.io/projected/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-kube-api-access-ckdr5\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.686291 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.686240    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-ta-internal\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.787385 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.787342    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.787577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.787397    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckdr5\" (UniqueName: \"kubernetes.io/projected/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-kube-api-access-ckdr5\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.787577 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.787448    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-ta-internal\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.788154 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.788129    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-ta-internal\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.789955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.789934    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.810588 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.810530    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckdr5\" (UniqueName: \"kubernetes.io/projected/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-kube-api-access-ckdr5\") pod \"data-science-collector-targetallocator-576f87968d-hctps\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:25.866428 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:25.866398    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:26.163524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:26.163494    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"]
Oct 30 18:55:26.167464 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:55:26.167439    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93e7afb2_c3cd_48e5_9ddb_1dc868cef6ce.slice/crio-72c41dc34a692ae2f581a23a1c8f23d68791f086b9790d988df6c5ad28e30ab5 WatchSource:0}: Error finding container 72c41dc34a692ae2f581a23a1c8f23d68791f086b9790d988df6c5ad28e30ab5: Status 404 returned error can't find the container with id 72c41dc34a692ae2f581a23a1c8f23d68791f086b9790d988df6c5ad28e30ab5
Oct 30 18:55:26.169583 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:26.169547    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:55:26.169667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:26.169631    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:55:26.236366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:26.236332    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps" event={"ID":"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce","Type":"ContainerStarted","Data":"a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800"}
Oct 30 18:55:26.236529 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:26.236377    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps" event={"ID":"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce","Type":"ContainerStarted","Data":"72c41dc34a692ae2f581a23a1c8f23d68791f086b9790d988df6c5ad28e30ab5"}
Oct 30 18:55:26.236600 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:26.236529    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:26.237797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:26.237768    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps" podUID="93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.160:8080/readyz\": dial tcp 10.133.0.160:8080: connect: connection refused"
Oct 30 18:55:26.277546 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:26.277496    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps" podStartSLOduration=1.277482232 podStartE2EDuration="1.277482232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:26.276494091 +0000 UTC m=+3496.443312237" watchObservedRunningTime="2026-10-30 18:55:26.277482232 +0000 UTC m=+3496.444300417"
Oct 30 18:55:27.241004 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.240978    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:55:27.311416 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.311384    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"]
Oct 30 18:55:27.311620 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.311591    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59" podUID="96789eae-245f-48eb-81af-7a4f8c7bd831" containerName="ta-container" containerID="cri-o://499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5" gracePeriod=30
Oct 30 18:55:27.561762 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.561737    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:27.704080 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.704054    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96789eae-245f-48eb-81af-7a4f8c7bd831-ta-internal\" (UniqueName: \"kubernetes.io/configmap/96789eae-245f-48eb-81af-7a4f8c7bd831-ta-internal\") pod \"96789eae-245f-48eb-81af-7a4f8c7bd831\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") "
Oct 30 18:55:27.704249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.704105    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96789eae-245f-48eb-81af-7a4f8c7bd831-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/96789eae-245f-48eb-81af-7a4f8c7bd831-data-science-collector-ta-server-cert\") pod \"96789eae-245f-48eb-81af-7a4f8c7bd831\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") "
Oct 30 18:55:27.704249 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.704129    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96789eae-245f-48eb-81af-7a4f8c7bd831-kube-api-access-plm6x\" (UniqueName: \"kubernetes.io/projected/96789eae-245f-48eb-81af-7a4f8c7bd831-kube-api-access-plm6x\") pod \"96789eae-245f-48eb-81af-7a4f8c7bd831\" (UID: \"96789eae-245f-48eb-81af-7a4f8c7bd831\") "
Oct 30 18:55:27.704497 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.704468    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96789eae-245f-48eb-81af-7a4f8c7bd831-ta-internal" pod "96789eae-245f-48eb-81af-7a4f8c7bd831" (UID: "96789eae-245f-48eb-81af-7a4f8c7bd831"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:55:27.706264 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.706236    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96789eae-245f-48eb-81af-7a4f8c7bd831-data-science-collector-ta-server-cert" pod "96789eae-245f-48eb-81af-7a4f8c7bd831" (UID: "96789eae-245f-48eb-81af-7a4f8c7bd831"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:55:27.706384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.706344    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96789eae-245f-48eb-81af-7a4f8c7bd831-kube-api-access-plm6x" pod "96789eae-245f-48eb-81af-7a4f8c7bd831" (UID: "96789eae-245f-48eb-81af-7a4f8c7bd831"). InnerVolumeSpecName "kube-api-access-plm6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:27.804896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.804827    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plm6x\" (UniqueName: \"kubernetes.io/projected/96789eae-245f-48eb-81af-7a4f8c7bd831-kube-api-access-plm6x\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:55:27.804896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.804852    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/96789eae-245f-48eb-81af-7a4f8c7bd831-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:55:27.804896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:27.804865    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/96789eae-245f-48eb-81af-7a4f8c7bd831-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:55:28.244843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.244806    5746 generic.go:356] "Generic (PLEG): container finished" podID="96789eae-245f-48eb-81af-7a4f8c7bd831" containerID="499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5" exitCode=0
Oct 30 18:55:28.245368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.244853    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"
Oct 30 18:55:28.245368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.244900    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59" event={"ID":"96789eae-245f-48eb-81af-7a4f8c7bd831","Type":"ContainerDied","Data":"499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5"}
Oct 30 18:55:28.245368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.244976    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59" event={"ID":"96789eae-245f-48eb-81af-7a4f8c7bd831","Type":"ContainerDied","Data":"e6608cab457757399e93688887a339d4b76d90a198118a6fe47d800e6c34eafc"}
Oct 30 18:55:28.245368 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.245004    5746 scope.go:122] "RemoveContainer" containerID="499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5"
Oct 30 18:55:28.253001 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.252981    5746 scope.go:122] "RemoveContainer" containerID="499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5"
Oct 30 18:55:28.253281 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:55:28.253264    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5\": container with ID starting with 499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5 not found: ID does not exist" containerID="499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5"
Oct 30 18:55:28.253353 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.253289    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5"} err="failed to get container status \"499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5\": rpc error: code = NotFound desc = could not find container \"499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5\": container with ID starting with 499ee3fe7f0281cb7ed8ffe3e373de521ddffcf9a09346af185fee836f79bce5 not found: ID does not exist"
Oct 30 18:55:28.293366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.293341    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"]
Oct 30 18:55:28.305079 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:28.305051    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-64df9b87d9-thv59"]
Oct 30 18:55:30.007859 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:30.007824    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96789eae-245f-48eb-81af-7a4f8c7bd831" path="/var/lib/kubelet/pods/96789eae-245f-48eb-81af-7a4f8c7bd831/volumes"
Oct 30 18:55:30.140544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:30.140505    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:55:40.140698 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:40.140659    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:55:42.790962 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:55:42.790927    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:55:50.140252 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:55:50.140212    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:00.140635 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.140598    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:00.694338 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.694298    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"]
Oct 30 18:56:00.694772 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.694749    5746 cpu_manager.go:414] "containerMap: removing container" podUID="96789eae-245f-48eb-81af-7a4f8c7bd831" containerName="ta-container"
Oct 30 18:56:00.694772 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.694773    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96789eae-245f-48eb-81af-7a4f8c7bd831" containerName="ta-container"
Oct 30 18:56:00.694994 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.694839    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="96789eae-245f-48eb-81af-7a4f8c7bd831" containerName="ta-container"
Oct 30 18:56:00.698003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.697975    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.725361 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.725333    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"]
Oct 30 18:56:00.771459 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.771426    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdvjc\" (UniqueName: \"kubernetes.io/projected/732f3980-29d0-415f-967b-8177131ac2b4-kube-api-access-hdvjc\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.771584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.771476    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/732f3980-29d0-415f-967b-8177131ac2b4-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.771584 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.771539    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/732f3980-29d0-415f-967b-8177131ac2b4-ta-internal\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.872409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.872379    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/732f3980-29d0-415f-967b-8177131ac2b4-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.872550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.872420    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/732f3980-29d0-415f-967b-8177131ac2b4-ta-internal\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.872550 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.872472    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdvjc\" (UniqueName: \"kubernetes.io/projected/732f3980-29d0-415f-967b-8177131ac2b4-kube-api-access-hdvjc\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.873136 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.873106    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/732f3980-29d0-415f-967b-8177131ac2b4-ta-internal\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.875142 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.875121    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/732f3980-29d0-415f-967b-8177131ac2b4-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:00.895121 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:00.895099    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdvjc\" (UniqueName: \"kubernetes.io/projected/732f3980-29d0-415f-967b-8177131ac2b4-kube-api-access-hdvjc\") pod \"data-science-collector-targetallocator-859bf5899f-q4jtb\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:01.009555 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.009473    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:01.305126 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.305026    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"]
Oct 30 18:56:01.309232 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:56:01.309203    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod732f3980_29d0_415f_967b_8177131ac2b4.slice/crio-07d38b5bd2c14fa741ca1501e72dfccef37daf77f29e2f34e3fa4c8cc0b3576c WatchSource:0}: Error finding container 07d38b5bd2c14fa741ca1501e72dfccef37daf77f29e2f34e3fa4c8cc0b3576c: Status 404 returned error can't find the container with id 07d38b5bd2c14fa741ca1501e72dfccef37daf77f29e2f34e3fa4c8cc0b3576c
Oct 30 18:56:01.312699 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.312661    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:56:01.312823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.312728    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:56:01.369774 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.369750    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb" event={"ID":"732f3980-29d0-415f-967b-8177131ac2b4","Type":"ContainerStarted","Data":"3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149"}
Oct 30 18:56:01.369892 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.369787    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb" event={"ID":"732f3980-29d0-415f-967b-8177131ac2b4","Type":"ContainerStarted","Data":"07d38b5bd2c14fa741ca1501e72dfccef37daf77f29e2f34e3fa4c8cc0b3576c"}
Oct 30 18:56:01.370054 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.369934    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:01.371144 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.371120    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb" podUID="732f3980-29d0-415f-967b-8177131ac2b4" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.161:8080/readyz\": dial tcp 10.133.0.161:8080: connect: connection refused"
Oct 30 18:56:01.409398 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:01.409354    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb" podStartSLOduration=1.409342111 podStartE2EDuration="1.409342111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:01.409111682 +0000 UTC m=+3531.575929834" watchObservedRunningTime="2026-10-30 18:56:01.409342111 +0000 UTC m=+3531.576160216"
Oct 30 18:56:02.374304 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.374278    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:02.444027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.443992    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"]
Oct 30 18:56:02.444209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.444187    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps" podUID="93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" containerName="ta-container" containerID="cri-o://a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800" gracePeriod=30
Oct 30 18:56:02.702281 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.702256    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:56:02.792392 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.792366    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-ta-internal\" (UniqueName: \"kubernetes.io/configmap/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-ta-internal\") pod \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") "
Oct 30 18:56:02.792574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.792404    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-kube-api-access-ckdr5\" (UniqueName: \"kubernetes.io/projected/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-kube-api-access-ckdr5\") pod \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") "
Oct 30 18:56:02.792574 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.792470    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-data-science-collector-ta-server-cert\") pod \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\" (UID: \"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce\") "
Oct 30 18:56:02.792746 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.792722    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-ta-internal" pod "93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" (UID: "93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:02.794740 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.794710    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-kube-api-access-ckdr5" pod "93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" (UID: "93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce"). InnerVolumeSpecName "kube-api-access-ckdr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:02.794858 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.794716    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-data-science-collector-ta-server-cert" pod "93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" (UID: "93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:02.893645 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.893604    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:02.893645 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.893642    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckdr5\" (UniqueName: \"kubernetes.io/projected/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-kube-api-access-ckdr5\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:02.893645 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:02.893652    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:03.378472 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.378433    5746 generic.go:356] "Generic (PLEG): container finished" podID="93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" containerID="a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800" exitCode=0
Oct 30 18:56:03.378953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.378483    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"
Oct 30 18:56:03.378953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.378514    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps" event={"ID":"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce","Type":"ContainerDied","Data":"a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800"}
Oct 30 18:56:03.378953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.378556    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps" event={"ID":"93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce","Type":"ContainerDied","Data":"72c41dc34a692ae2f581a23a1c8f23d68791f086b9790d988df6c5ad28e30ab5"}
Oct 30 18:56:03.378953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.378572    5746 scope.go:122] "RemoveContainer" containerID="a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800"
Oct 30 18:56:03.387089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.387071    5746 scope.go:122] "RemoveContainer" containerID="a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800"
Oct 30 18:56:03.387336 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:56:03.387321    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800\": container with ID starting with a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800 not found: ID does not exist" containerID="a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800"
Oct 30 18:56:03.387397 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.387343    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800"} err="failed to get container status \"a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800\": rpc error: code = NotFound desc = could not find container \"a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800\": container with ID starting with a80fbe30a828359a244327d8d1ae1a6c3100849ae9c8edc62627c410bee82800 not found: ID does not exist"
Oct 30 18:56:03.429997 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.429971    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"]
Oct 30 18:56:03.436132 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:03.436111    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-576f87968d-hctps"]
Oct 30 18:56:04.007536 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:04.007505    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" path="/var/lib/kubelet/pods/93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce/volumes"
Oct 30 18:56:10.140406 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.140371    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:10.457185 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.457097    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"]
Oct 30 18:56:10.457342 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.457303    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb" podUID="732f3980-29d0-415f-967b-8177131ac2b4" containerName="ta-container" containerID="cri-o://3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149" gracePeriod=30
Oct 30 18:56:10.527122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.526718    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:56:10.527122 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.526941    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="44e6260e-6dd0-4b5d-a028-61066f52dd7e" containerName="perses" containerID="cri-o://8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f" gracePeriod=30
Oct 30 18:56:10.844262 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.844240    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:10.847320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.847268    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:10.855218 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855196    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/732f3980-29d0-415f-967b-8177131ac2b4-ta-internal\" (UniqueName: \"kubernetes.io/configmap/732f3980-29d0-415f-967b-8177131ac2b4-ta-internal\") pod \"732f3980-29d0-415f-967b-8177131ac2b4\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") "
Oct 30 18:56:10.855331 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855247    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/732f3980-29d0-415f-967b-8177131ac2b4-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/732f3980-29d0-415f-967b-8177131ac2b4-data-science-collector-ta-server-cert\") pod \"732f3980-29d0-415f-967b-8177131ac2b4\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") "
Oct 30 18:56:10.855405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855346    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") "
Oct 30 18:56:10.855465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855411    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/732f3980-29d0-415f-967b-8177131ac2b4-kube-api-access-hdvjc\" (UniqueName: \"kubernetes.io/projected/732f3980-29d0-415f-967b-8177131ac2b4-kube-api-access-hdvjc\") pod \"732f3980-29d0-415f-967b-8177131ac2b4\" (UID: \"732f3980-29d0-415f-967b-8177131ac2b4\") "
Oct 30 18:56:10.855465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855441    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44e6260e-6dd0-4b5d-a028-61066f52dd7e-kube-api-access-7mph2\" (UniqueName: \"kubernetes.io/projected/44e6260e-6dd0-4b5d-a028-61066f52dd7e-kube-api-access-7mph2\") pod \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") "
Oct 30 18:56:10.855570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855484    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44e6260e-6dd0-4b5d-a028-61066f52dd7e-config\" (UniqueName: \"kubernetes.io/configmap/44e6260e-6dd0-4b5d-a028-61066f52dd7e-config\") pod \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") "
Oct 30 18:56:10.855570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855510    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/732f3980-29d0-415f-967b-8177131ac2b4-ta-internal" pod "732f3980-29d0-415f-967b-8177131ac2b4" (UID: "732f3980-29d0-415f-967b-8177131ac2b4"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:10.855570 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855533    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44e6260e-6dd0-4b5d-a028-61066f52dd7e-plugins\" (UniqueName: \"kubernetes.io/empty-dir/44e6260e-6dd0-4b5d-a028-61066f52dd7e-plugins\") pod \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\" (UID: \"44e6260e-6dd0-4b5d-a028-61066f52dd7e\") "
Oct 30 18:56:10.855783 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855752    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/732f3980-29d0-415f-967b-8177131ac2b4-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:10.856019 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.855871    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44e6260e-6dd0-4b5d-a028-61066f52dd7e-config" pod "44e6260e-6dd0-4b5d-a028-61066f52dd7e" (UID: "44e6260e-6dd0-4b5d-a028-61066f52dd7e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:10.858270 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.858224    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/732f3980-29d0-415f-967b-8177131ac2b4-data-science-collector-ta-server-cert" pod "732f3980-29d0-415f-967b-8177131ac2b4" (UID: "732f3980-29d0-415f-967b-8177131ac2b4"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:10.858270 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.858222    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/732f3980-29d0-415f-967b-8177131ac2b4-kube-api-access-hdvjc" pod "732f3980-29d0-415f-967b-8177131ac2b4" (UID: "732f3980-29d0-415f-967b-8177131ac2b4"). InnerVolumeSpecName "kube-api-access-hdvjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:10.858514 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.858493    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "44e6260e-6dd0-4b5d-a028-61066f52dd7e" (UID: "44e6260e-6dd0-4b5d-a028-61066f52dd7e"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:56:10.858708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.858689    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e6260e-6dd0-4b5d-a028-61066f52dd7e-kube-api-access-7mph2" pod "44e6260e-6dd0-4b5d-a028-61066f52dd7e" (UID: "44e6260e-6dd0-4b5d-a028-61066f52dd7e"). InnerVolumeSpecName "kube-api-access-7mph2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:10.956242 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.956211    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/732f3980-29d0-415f-967b-8177131ac2b4-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:10.956405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.956254    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:56:10.956405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.956270    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdvjc\" (UniqueName: \"kubernetes.io/projected/732f3980-29d0-415f-967b-8177131ac2b4-kube-api-access-hdvjc\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:10.956405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.956279    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mph2\" (UniqueName: \"kubernetes.io/projected/44e6260e-6dd0-4b5d-a028-61066f52dd7e-kube-api-access-7mph2\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:10.956405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.956288    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44e6260e-6dd0-4b5d-a028-61066f52dd7e-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:10.968995 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:10.968976    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:56:11.008978 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.008889    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e6260e-6dd0-4b5d-a028-61066f52dd7e-plugins" pod "44e6260e-6dd0-4b5d-a028-61066f52dd7e" (UID: "44e6260e-6dd0-4b5d-a028-61066f52dd7e"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:11.057200 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.057166    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/44e6260e-6dd0-4b5d-a028-61066f52dd7e-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:11.057200 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.057196    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:56:11.418150 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.418119    5746 generic.go:356] "Generic (PLEG): container finished" podID="732f3980-29d0-415f-967b-8177131ac2b4" containerID="3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149" exitCode=0
Oct 30 18:56:11.418743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.418175    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"
Oct 30 18:56:11.418743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.418178    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb" event={"ID":"732f3980-29d0-415f-967b-8177131ac2b4","Type":"ContainerDied","Data":"3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149"}
Oct 30 18:56:11.418743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.418284    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb" event={"ID":"732f3980-29d0-415f-967b-8177131ac2b4","Type":"ContainerDied","Data":"07d38b5bd2c14fa741ca1501e72dfccef37daf77f29e2f34e3fa4c8cc0b3576c"}
Oct 30 18:56:11.418743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.418304    5746 scope.go:122] "RemoveContainer" containerID="3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149"
Oct 30 18:56:11.419743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.419705    5746 generic.go:356] "Generic (PLEG): container finished" podID="44e6260e-6dd0-4b5d-a028-61066f52dd7e" containerID="8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f" exitCode=0
Oct 30 18:56:11.419743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.419742    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:11.419955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.419850    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"44e6260e-6dd0-4b5d-a028-61066f52dd7e","Type":"ContainerDied","Data":"8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f"}
Oct 30 18:56:11.419955 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.419890    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"44e6260e-6dd0-4b5d-a028-61066f52dd7e","Type":"ContainerDied","Data":"03e3e08172349ee5e9a89da50e304f67a25ce8ff36ec6d8f8362b1d980b0455b"}
Oct 30 18:56:11.427522 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.427492    5746 scope.go:122] "RemoveContainer" containerID="3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149"
Oct 30 18:56:11.427866 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:56:11.427846    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149\": container with ID starting with 3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149 not found: ID does not exist" containerID="3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149"
Oct 30 18:56:11.427984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.427876    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149"} err="failed to get container status \"3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149\": rpc error: code = NotFound desc = could not find container \"3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149\": container with ID starting with 3c2edab652a1e6698b56920e97d89c7785b81765afb4fb6d530064234996c149 not found: ID does not exist"
Oct 30 18:56:11.427984 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.427898    5746 scope.go:122] "RemoveContainer" containerID="8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f"
Oct 30 18:56:11.437281 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.437262    5746 scope.go:122] "RemoveContainer" containerID="8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f"
Oct 30 18:56:11.437598 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:56:11.437581    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f\": container with ID starting with 8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f not found: ID does not exist" containerID="8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f"
Oct 30 18:56:11.437677 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.437605    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f"} err="failed to get container status \"8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f\": rpc error: code = NotFound desc = could not find container \"8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f\": container with ID starting with 8b3b9eb58d4f28515f59ad5e0c40c68ccfed60bf949b4b595dd36b8a6e1c290f not found: ID does not exist"
Oct 30 18:56:11.467775 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.467745    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"]
Oct 30 18:56:11.472208 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.472185    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-859bf5899f-q4jtb"]
Oct 30 18:56:11.504975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.504947    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:56:11.509652 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:11.509628    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:56:12.008161 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:12.008130    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44e6260e-6dd0-4b5d-a028-61066f52dd7e" path="/var/lib/kubelet/pods/44e6260e-6dd0-4b5d-a028-61066f52dd7e/volumes"
Oct 30 18:56:12.008511 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:12.008498    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="732f3980-29d0-415f-967b-8177131ac2b4" path="/var/lib/kubelet/pods/732f3980-29d0-415f-967b-8177131ac2b4/volumes"
Oct 30 18:56:12.789845 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:56:12.789815    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:56:20.141033 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:20.140992    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:21.087776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.087743    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"]
Oct 30 18:56:21.088042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088030    5746 cpu_manager.go:414] "containerMap: removing container" podUID="732f3980-29d0-415f-967b-8177131ac2b4" containerName="ta-container"
Oct 30 18:56:21.088102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088044    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="732f3980-29d0-415f-967b-8177131ac2b4" containerName="ta-container"
Oct 30 18:56:21.088102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088058    5746 cpu_manager.go:414] "containerMap: removing container" podUID="93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" containerName="ta-container"
Oct 30 18:56:21.088102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088063    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" containerName="ta-container"
Oct 30 18:56:21.088102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088085    5746 cpu_manager.go:414] "containerMap: removing container" podUID="44e6260e-6dd0-4b5d-a028-61066f52dd7e" containerName="perses"
Oct 30 18:56:21.088102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088091    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44e6260e-6dd0-4b5d-a028-61066f52dd7e" containerName="perses"
Oct 30 18:56:21.088387 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088137    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="44e6260e-6dd0-4b5d-a028-61066f52dd7e" containerName="perses"
Oct 30 18:56:21.088387 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088145    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="732f3980-29d0-415f-967b-8177131ac2b4" containerName="ta-container"
Oct 30 18:56:21.088387 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.088151    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="93e7afb2-c3cd-48e5-9ddb-1dc868cef6ce" containerName="ta-container"
Oct 30 18:56:21.110625 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.110589    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"]
Oct 30 18:56:21.110750 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.110694    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"
Oct 30 18:56:21.125833 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.125798    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmj2d\" (UniqueName: \"kubernetes.io/projected/9894daef-6efb-44f7-a7ac-20815b1df8c0-kube-api-access-lmj2d\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj\" (UID: \"9894daef-6efb-44f7-a7ac-20815b1df8c0\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"
Oct 30 18:56:21.137344 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.137322    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:56:21.226291 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.226255    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmj2d\" (UniqueName: \"kubernetes.io/projected/9894daef-6efb-44f7-a7ac-20815b1df8c0-kube-api-access-lmj2d\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj\" (UID: \"9894daef-6efb-44f7-a7ac-20815b1df8c0\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"
Oct 30 18:56:21.251594 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.251566    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmj2d\" (UniqueName: \"kubernetes.io/projected/9894daef-6efb-44f7-a7ac-20815b1df8c0-kube-api-access-lmj2d\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj\" (UID: \"9894daef-6efb-44f7-a7ac-20815b1df8c0\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"
Oct 30 18:56:21.422657 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.422623    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"
Oct 30 18:56:21.739534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:21.739445    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"]
Oct 30 18:56:21.743263 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:56:21.743235    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9894daef_6efb_44f7_a7ac_20815b1df8c0.slice/crio-93f518f4b9421e566ece561b7830024cb7d9e0f9eb2f135030f4124711a279c3 WatchSource:0}: Error finding container 93f518f4b9421e566ece561b7830024cb7d9e0f9eb2f135030f4124711a279c3: Status 404 returned error can't find the container with id 93f518f4b9421e566ece561b7830024cb7d9e0f9eb2f135030f4124711a279c3
Oct 30 18:56:22.139094 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.139055    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:56:22.142337 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.142318    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.164099 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.164075    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:56:22.206559 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.206524    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:56:22.221491 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.221455    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"]
Oct 30 18:56:22.225369 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.225350    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.243696 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.243670    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"]
Oct 30 18:56:22.256132 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.256099    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:56:22.256132 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.256114    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-5c8ng\""
Oct 30 18:56:22.306649 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.306628    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:56:22.336013 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.335976    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tx7z\" (UniqueName: \"kubernetes.io/projected/e9aac573-36ba-40f9-8620-69db2b46ec77-kube-api-access-2tx7z\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.336134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.336030    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/51bb24b3-472b-4a94-924f-1078a8c222e8-config\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.336134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.336116    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2s95\" (UniqueName: \"kubernetes.io/projected/51bb24b3-472b-4a94-924f-1078a8c222e8-kube-api-access-k2s95\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.336209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.336151    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/51bb24b3-472b-4a94-924f-1078a8c222e8-plugins\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.336209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.336173    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/e9aac573-36ba-40f9-8620-69db2b46ec77-ta-internal\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.336283 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.336224    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e9aac573-36ba-40f9-8620-69db2b46ec77-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.437592 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.437510    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tx7z\" (UniqueName: \"kubernetes.io/projected/e9aac573-36ba-40f9-8620-69db2b46ec77-kube-api-access-2tx7z\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.437592 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.437564    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/51bb24b3-472b-4a94-924f-1078a8c222e8-config\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.437797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.437599    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2s95\" (UniqueName: \"kubernetes.io/projected/51bb24b3-472b-4a94-924f-1078a8c222e8-kube-api-access-k2s95\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.437797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.437638    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/51bb24b3-472b-4a94-924f-1078a8c222e8-plugins\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.437797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.437665    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/e9aac573-36ba-40f9-8620-69db2b46ec77-ta-internal\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.437797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.437692    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e9aac573-36ba-40f9-8620-69db2b46ec77-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.438155 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.438130    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/51bb24b3-472b-4a94-924f-1078a8c222e8-plugins\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.438370 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.438348    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/51bb24b3-472b-4a94-924f-1078a8c222e8-config\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.438616 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.438379    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/e9aac573-36ba-40f9-8620-69db2b46ec77-ta-internal\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.440280 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.440260    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e9aac573-36ba-40f9-8620-69db2b46ec77-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.460377 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.460349    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tx7z\" (UniqueName: \"kubernetes.io/projected/e9aac573-36ba-40f9-8620-69db2b46ec77-kube-api-access-2tx7z\") pod \"data-science-collector-targetallocator-6d54d49dd9-vgmzp\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.460580 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.460560    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2s95\" (UniqueName: \"kubernetes.io/projected/51bb24b3-472b-4a94-924f-1078a8c222e8-kube-api-access-k2s95\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:22.469000 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.468965    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj" event={"ID":"9894daef-6efb-44f7-a7ac-20815b1df8c0","Type":"ContainerStarted","Data":"f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53"}
Oct 30 18:56:22.469167 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.469057    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj" event={"ID":"9894daef-6efb-44f7-a7ac-20815b1df8c0","Type":"ContainerStarted","Data":"93f518f4b9421e566ece561b7830024cb7d9e0f9eb2f135030f4124711a279c3"}
Oct 30 18:56:22.507528 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.507475    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj" podStartSLOduration=1.5074614899999998 podStartE2EDuration="1.50746149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:22.506644119 +0000 UTC m=+3552.673462263" watchObservedRunningTime="2026-10-30 18:56:22.50746149 +0000 UTC m=+3552.674279617"
Oct 30 18:56:22.536540 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.536502    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:22.832662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.832579    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"]
Oct 30 18:56:22.836219 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:56:22.836191    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9aac573_36ba_40f9_8620_69db2b46ec77.slice/crio-9e643d80909769123b9cf6e41f59609fc172cf65c3846ba6729ecfa5066c7aae WatchSource:0}: Error finding container 9e643d80909769123b9cf6e41f59609fc172cf65c3846ba6729ecfa5066c7aae: Status 404 returned error can't find the container with id 9e643d80909769123b9cf6e41f59609fc172cf65c3846ba6729ecfa5066c7aae
Oct 30 18:56:22.838341 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.838297    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:56:22.838436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:22.838367    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:56:23.474681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:23.474648    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp" event={"ID":"e9aac573-36ba-40f9-8620-69db2b46ec77","Type":"ContainerStarted","Data":"b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590"}
Oct 30 18:56:23.474681 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:23.474692    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp" event={"ID":"e9aac573-36ba-40f9-8620-69db2b46ec77","Type":"ContainerStarted","Data":"9e643d80909769123b9cf6e41f59609fc172cf65c3846ba6729ecfa5066c7aae"}
Oct 30 18:56:23.475267 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:23.475012    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:23.476736 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:23.476714    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:23.523715 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:23.523652    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp" podStartSLOduration=1.523635322 podStartE2EDuration="1.523635322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:23.522735805 +0000 UTC m=+3553.689553951" watchObservedRunningTime="2026-10-30 18:56:23.523635322 +0000 UTC m=+3553.690453462"
Oct 30 18:56:24.556836 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:24.556805    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:24.584549 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:24.584521    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:24.657443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:24.657402    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:24.657606 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:24.657547    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:24.671821 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:24.671795    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:24.774018 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:24.773990    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:24.803560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:24.803533    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:24.854060 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:24.854029    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:25.151268 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:25.151178    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:56:25.154484 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:56:25.154455    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51bb24b3_472b_4a94_924f_1078a8c222e8.slice/crio-f63a40722eacab93e2526ac87140e844b86871ab92f6d94176cc742a5b9876cf WatchSource:0}: Error finding container f63a40722eacab93e2526ac87140e844b86871ab92f6d94176cc742a5b9876cf: Status 404 returned error can't find the container with id f63a40722eacab93e2526ac87140e844b86871ab92f6d94176cc742a5b9876cf
Oct 30 18:56:25.492802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:25.492714    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"51bb24b3-472b-4a94-924f-1078a8c222e8","Type":"ContainerStarted","Data":"0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc"}
Oct 30 18:56:25.492802 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:25.492762    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"51bb24b3-472b-4a94-924f-1078a8c222e8","Type":"ContainerStarted","Data":"f63a40722eacab93e2526ac87140e844b86871ab92f6d94176cc742a5b9876cf"}
Oct 30 18:56:25.493053 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:25.492818    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:25.533344 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:25.533280    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=3.533264095 podStartE2EDuration="3.533264095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:25.532303373 +0000 UTC m=+3555.699121524" watchObservedRunningTime="2026-10-30 18:56:25.533264095 +0000 UTC m=+3555.700082221"
Oct 30 18:56:30.140175 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:30.140136    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:30.667405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:30.667373    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"]
Oct 30 18:56:30.667578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:30.667542    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj" podUID="9894daef-6efb-44f7-a7ac-20815b1df8c0" containerName="thanos-querier" containerID="cri-o://f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53" gracePeriod=30
Oct 30 18:56:30.912568 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:30.912541    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"
Oct 30 18:56:31.016595 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.016513    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9894daef-6efb-44f7-a7ac-20815b1df8c0-kube-api-access-lmj2d\" (UniqueName: \"kubernetes.io/projected/9894daef-6efb-44f7-a7ac-20815b1df8c0-kube-api-access-lmj2d\") pod \"9894daef-6efb-44f7-a7ac-20815b1df8c0\" (UID: \"9894daef-6efb-44f7-a7ac-20815b1df8c0\") "
Oct 30 18:56:31.018748 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.018716    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9894daef-6efb-44f7-a7ac-20815b1df8c0-kube-api-access-lmj2d" pod "9894daef-6efb-44f7-a7ac-20815b1df8c0" (UID: "9894daef-6efb-44f7-a7ac-20815b1df8c0"). InnerVolumeSpecName "kube-api-access-lmj2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:31.117264 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.117237    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmj2d\" (UniqueName: \"kubernetes.io/projected/9894daef-6efb-44f7-a7ac-20815b1df8c0-kube-api-access-lmj2d\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:31.516140 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.516105    5746 generic.go:356] "Generic (PLEG): container finished" podID="9894daef-6efb-44f7-a7ac-20815b1df8c0" containerID="f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53" exitCode=0
Oct 30 18:56:31.516625 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.516146    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"
Oct 30 18:56:31.516625 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.516186    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj" event={"ID":"9894daef-6efb-44f7-a7ac-20815b1df8c0","Type":"ContainerDied","Data":"f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53"}
Oct 30 18:56:31.516625 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.516231    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj" event={"ID":"9894daef-6efb-44f7-a7ac-20815b1df8c0","Type":"ContainerDied","Data":"93f518f4b9421e566ece561b7830024cb7d9e0f9eb2f135030f4124711a279c3"}
Oct 30 18:56:31.516625 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.516248    5746 scope.go:122] "RemoveContainer" containerID="f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53"
Oct 30 18:56:31.524876 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.524862    5746 scope.go:122] "RemoveContainer" containerID="f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53"
Oct 30 18:56:31.525174 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:56:31.525159    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53\": container with ID starting with f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53 not found: ID does not exist" containerID="f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53"
Oct 30 18:56:31.525230 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.525182    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53"} err="failed to get container status \"f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53\": rpc error: code = NotFound desc = could not find container \"f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53\": container with ID starting with f53c4fe82865bafe928a11570e719e374ca4b308c99a7136fa0e855037d37c53 not found: ID does not exist"
Oct 30 18:56:31.564298 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.564276    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"]
Oct 30 18:56:31.576047 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:31.576019    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-5vnhj"]
Oct 30 18:56:32.008726 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:32.008692    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9894daef-6efb-44f7-a7ac-20815b1df8c0" path="/var/lib/kubelet/pods/9894daef-6efb-44f7-a7ac-20815b1df8c0/volumes"
Oct 30 18:56:36.499456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:36.499423    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:56:40.140296 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:40.140256    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:42.789966 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:56:42.789932    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:56:50.140651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:50.140613    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:53.760688 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.760651    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"]
Oct 30 18:56:53.761127 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.760964    5746 cpu_manager.go:414] "containerMap: removing container" podUID="9894daef-6efb-44f7-a7ac-20815b1df8c0" containerName="thanos-querier"
Oct 30 18:56:53.761127 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.760978    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9894daef-6efb-44f7-a7ac-20815b1df8c0" containerName="thanos-querier"
Oct 30 18:56:53.761127 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.761072    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="9894daef-6efb-44f7-a7ac-20815b1df8c0" containerName="thanos-querier"
Oct 30 18:56:53.770099 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.770070    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:53.786286 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.786264    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"]
Oct 30 18:56:53.915260 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.915227    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ca872639-3127-4885-ac9e-e92b0f9aa4dc-ta-internal\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:53.915454 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.915272    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8qwl\" (UniqueName: \"kubernetes.io/projected/ca872639-3127-4885-ac9e-e92b0f9aa4dc-kube-api-access-c8qwl\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:53.915454 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:53.915371    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ca872639-3127-4885-ac9e-e92b0f9aa4dc-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.015860 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.015784    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ca872639-3127-4885-ac9e-e92b0f9aa4dc-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.015860 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.015845    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ca872639-3127-4885-ac9e-e92b0f9aa4dc-ta-internal\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.016062 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.015870    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8qwl\" (UniqueName: \"kubernetes.io/projected/ca872639-3127-4885-ac9e-e92b0f9aa4dc-kube-api-access-c8qwl\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.016592 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.016569    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ca872639-3127-4885-ac9e-e92b0f9aa4dc-ta-internal\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.018407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.018388    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ca872639-3127-4885-ac9e-e92b0f9aa4dc-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.038620 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.038599    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8qwl\" (UniqueName: \"kubernetes.io/projected/ca872639-3127-4885-ac9e-e92b0f9aa4dc-kube-api-access-c8qwl\") pod \"data-science-collector-targetallocator-6c8476f944-k4g65\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.080692 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.080669    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.374950 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.374920    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"]
Oct 30 18:56:54.378700 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:56:54.378674    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca872639_3127_4885_ac9e_e92b0f9aa4dc.slice/crio-c3bc31f4fd5af70e2f4d1acc5cfa323e983c503165ab1540478a8bc82e0ff9d3 WatchSource:0}: Error finding container c3bc31f4fd5af70e2f4d1acc5cfa323e983c503165ab1540478a8bc82e0ff9d3: Status 404 returned error can't find the container with id c3bc31f4fd5af70e2f4d1acc5cfa323e983c503165ab1540478a8bc82e0ff9d3
Oct 30 18:56:54.381211 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.381177    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:56:54.381305 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.381235    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:56:54.605610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.605571    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65" event={"ID":"ca872639-3127-4885-ac9e-e92b0f9aa4dc","Type":"ContainerStarted","Data":"9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313"}
Oct 30 18:56:54.605610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.605610    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65" event={"ID":"ca872639-3127-4885-ac9e-e92b0f9aa4dc","Type":"ContainerStarted","Data":"c3bc31f4fd5af70e2f4d1acc5cfa323e983c503165ab1540478a8bc82e0ff9d3"}
Oct 30 18:56:54.605864 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.605784    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:54.606937 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.606895    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65" podUID="ca872639-3127-4885-ac9e-e92b0f9aa4dc" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.165:8080/readyz\": dial tcp 10.133.0.165:8080: connect: connection refused"
Oct 30 18:56:54.663363 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:54.663254    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65" podStartSLOduration=1.663240283 podStartE2EDuration="1.663240283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:54.66252486 +0000 UTC m=+3584.829342969" watchObservedRunningTime="2026-10-30 18:56:54.663240283 +0000 UTC m=+3584.830058410"
Oct 30 18:56:55.611573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:55.611546    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:56:55.682158 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:55.682129    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"]
Oct 30 18:56:55.682340 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:55.682298    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp" podUID="e9aac573-36ba-40f9-8620-69db2b46ec77" containerName="ta-container" containerID="cri-o://b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590" gracePeriod=30
Oct 30 18:56:55.938989 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:55.938964    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:56.032805 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.032775    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9aac573-36ba-40f9-8620-69db2b46ec77-kube-api-access-2tx7z\" (UniqueName: \"kubernetes.io/projected/e9aac573-36ba-40f9-8620-69db2b46ec77-kube-api-access-2tx7z\") pod \"e9aac573-36ba-40f9-8620-69db2b46ec77\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") "
Oct 30 18:56:56.032991 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.032860    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9aac573-36ba-40f9-8620-69db2b46ec77-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e9aac573-36ba-40f9-8620-69db2b46ec77-data-science-collector-ta-server-cert\") pod \"e9aac573-36ba-40f9-8620-69db2b46ec77\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") "
Oct 30 18:56:56.032991 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.032895    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9aac573-36ba-40f9-8620-69db2b46ec77-ta-internal\" (UniqueName: \"kubernetes.io/configmap/e9aac573-36ba-40f9-8620-69db2b46ec77-ta-internal\") pod \"e9aac573-36ba-40f9-8620-69db2b46ec77\" (UID: \"e9aac573-36ba-40f9-8620-69db2b46ec77\") "
Oct 30 18:56:56.033332 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.033305    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9aac573-36ba-40f9-8620-69db2b46ec77-ta-internal" pod "e9aac573-36ba-40f9-8620-69db2b46ec77" (UID: "e9aac573-36ba-40f9-8620-69db2b46ec77"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:56.035076 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.035047    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9aac573-36ba-40f9-8620-69db2b46ec77-kube-api-access-2tx7z" pod "e9aac573-36ba-40f9-8620-69db2b46ec77" (UID: "e9aac573-36ba-40f9-8620-69db2b46ec77"). InnerVolumeSpecName "kube-api-access-2tx7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:56.035163 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.035053    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9aac573-36ba-40f9-8620-69db2b46ec77-data-science-collector-ta-server-cert" pod "e9aac573-36ba-40f9-8620-69db2b46ec77" (UID: "e9aac573-36ba-40f9-8620-69db2b46ec77"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:56.133434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.133403    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/e9aac573-36ba-40f9-8620-69db2b46ec77-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:56.133434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.133428    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/e9aac573-36ba-40f9-8620-69db2b46ec77-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:56.133434 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.133439    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tx7z\" (UniqueName: \"kubernetes.io/projected/e9aac573-36ba-40f9-8620-69db2b46ec77-kube-api-access-2tx7z\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:56:56.615361 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.615330    5746 generic.go:356] "Generic (PLEG): container finished" podID="e9aac573-36ba-40f9-8620-69db2b46ec77" containerID="b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590" exitCode=0
Oct 30 18:56:56.615850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.615373    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"
Oct 30 18:56:56.615850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.615421    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp" event={"ID":"e9aac573-36ba-40f9-8620-69db2b46ec77","Type":"ContainerDied","Data":"b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590"}
Oct 30 18:56:56.615850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.615467    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp" event={"ID":"e9aac573-36ba-40f9-8620-69db2b46ec77","Type":"ContainerDied","Data":"9e643d80909769123b9cf6e41f59609fc172cf65c3846ba6729ecfa5066c7aae"}
Oct 30 18:56:56.615850 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.615492    5746 scope.go:122] "RemoveContainer" containerID="b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590"
Oct 30 18:56:56.623875 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.623858    5746 scope.go:122] "RemoveContainer" containerID="b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590"
Oct 30 18:56:56.624229 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:56:56.624213    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590\": container with ID starting with b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590 not found: ID does not exist" containerID="b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590"
Oct 30 18:56:56.624279 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.624236    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590"} err="failed to get container status \"b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590\": rpc error: code = NotFound desc = could not find container \"b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590\": container with ID starting with b225cc6e110a8a0e4ca555bb01668bbd2363c4966bce5eef4c1eff5ca6632590 not found: ID does not exist"
Oct 30 18:56:56.665060 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.665033    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"]
Oct 30 18:56:56.676571 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:56.676545    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6d54d49dd9-vgmzp"]
Oct 30 18:56:58.007859 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:56:58.007830    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9aac573-36ba-40f9-8620-69db2b46ec77" path="/var/lib/kubelet/pods/e9aac573-36ba-40f9-8620-69db2b46ec77/volumes"
Oct 30 18:57:00.140435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:00.140393    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:10.140648 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.140612    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:10.637557 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.637522    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"]
Oct 30 18:57:10.637949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.637931    5746 cpu_manager.go:414] "containerMap: removing container" podUID="e9aac573-36ba-40f9-8620-69db2b46ec77" containerName="ta-container"
Oct 30 18:57:10.638060 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.637952    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9aac573-36ba-40f9-8620-69db2b46ec77" containerName="ta-container"
Oct 30 18:57:10.638060 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.638037    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9aac573-36ba-40f9-8620-69db2b46ec77" containerName="ta-container"
Oct 30 18:57:10.643410 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.643388    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.662426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.662398    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"]
Oct 30 18:57:10.759265 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.759224    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/68298f80-c1ef-47a7-9ed7-f149e025076e-ta-internal\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.759465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.759275    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js9gj\" (UniqueName: \"kubernetes.io/projected/68298f80-c1ef-47a7-9ed7-f149e025076e-kube-api-access-js9gj\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.759465 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.759341    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/68298f80-c1ef-47a7-9ed7-f149e025076e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.860187 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.860154    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/68298f80-c1ef-47a7-9ed7-f149e025076e-ta-internal\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.860366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.860201    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js9gj\" (UniqueName: \"kubernetes.io/projected/68298f80-c1ef-47a7-9ed7-f149e025076e-kube-api-access-js9gj\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.860366 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.860226    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/68298f80-c1ef-47a7-9ed7-f149e025076e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.860861 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.860839    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/68298f80-c1ef-47a7-9ed7-f149e025076e-ta-internal\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.862876 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.862851    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/68298f80-c1ef-47a7-9ed7-f149e025076e-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.884117 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.884087    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js9gj\" (UniqueName: \"kubernetes.io/projected/68298f80-c1ef-47a7-9ed7-f149e025076e-kube-api-access-js9gj\") pod \"data-science-collector-targetallocator-74bf949bc9-g2smj\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:10.954267 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:10.954190    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:11.253147 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.253059    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"]
Oct 30 18:57:11.256450 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:57:11.256420    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68298f80_c1ef_47a7_9ed7_f149e025076e.slice/crio-596bb8879eb4222d8f1236fffc2ba21c5380036c0837db3305b2d1688ff37002 WatchSource:0}: Error finding container 596bb8879eb4222d8f1236fffc2ba21c5380036c0837db3305b2d1688ff37002: Status 404 returned error can't find the container with id 596bb8879eb4222d8f1236fffc2ba21c5380036c0837db3305b2d1688ff37002
Oct 30 18:57:11.258801 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.258765    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:57:11.258927 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.258838    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:57:11.679622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.679590    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj" event={"ID":"68298f80-c1ef-47a7-9ed7-f149e025076e","Type":"ContainerStarted","Data":"7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec"}
Oct 30 18:57:11.679622 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.679630    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj" event={"ID":"68298f80-c1ef-47a7-9ed7-f149e025076e","Type":"ContainerStarted","Data":"596bb8879eb4222d8f1236fffc2ba21c5380036c0837db3305b2d1688ff37002"}
Oct 30 18:57:11.679929 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.679783    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:11.681545 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.681523    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:11.720630 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.720585    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj" podStartSLOduration=1.72057202 podStartE2EDuration="1.72057202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:11.720059674 +0000 UTC m=+3601.886877811" watchObservedRunningTime="2026-10-30 18:57:11.72057202 +0000 UTC m=+3601.887390211"
Oct 30 18:57:11.789858 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.789827    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"]
Oct 30 18:57:11.790066 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:11.790041    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65" podUID="ca872639-3127-4885-ac9e-e92b0f9aa4dc" containerName="ta-container" containerID="cri-o://9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313" gracePeriod=30
Oct 30 18:57:12.049484 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.049458    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:57:12.169963 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.169895    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca872639-3127-4885-ac9e-e92b0f9aa4dc-kube-api-access-c8qwl\" (UniqueName: \"kubernetes.io/projected/ca872639-3127-4885-ac9e-e92b0f9aa4dc-kube-api-access-c8qwl\") pod \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") "
Oct 30 18:57:12.170149 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.169988    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca872639-3127-4885-ac9e-e92b0f9aa4dc-ta-internal\" (UniqueName: \"kubernetes.io/configmap/ca872639-3127-4885-ac9e-e92b0f9aa4dc-ta-internal\") pod \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") "
Oct 30 18:57:12.170149 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.170008    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca872639-3127-4885-ac9e-e92b0f9aa4dc-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ca872639-3127-4885-ac9e-e92b0f9aa4dc-data-science-collector-ta-server-cert\") pod \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\" (UID: \"ca872639-3127-4885-ac9e-e92b0f9aa4dc\") "
Oct 30 18:57:12.170426 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.170398    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca872639-3127-4885-ac9e-e92b0f9aa4dc-ta-internal" pod "ca872639-3127-4885-ac9e-e92b0f9aa4dc" (UID: "ca872639-3127-4885-ac9e-e92b0f9aa4dc"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:12.172236 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.172216    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca872639-3127-4885-ac9e-e92b0f9aa4dc-kube-api-access-c8qwl" pod "ca872639-3127-4885-ac9e-e92b0f9aa4dc" (UID: "ca872639-3127-4885-ac9e-e92b0f9aa4dc"). InnerVolumeSpecName "kube-api-access-c8qwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:12.172355 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.172333    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca872639-3127-4885-ac9e-e92b0f9aa4dc-data-science-collector-ta-server-cert" pod "ca872639-3127-4885-ac9e-e92b0f9aa4dc" (UID: "ca872639-3127-4885-ac9e-e92b0f9aa4dc"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:12.270615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.270530    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8qwl\" (UniqueName: \"kubernetes.io/projected/ca872639-3127-4885-ac9e-e92b0f9aa4dc-kube-api-access-c8qwl\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:12.270615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.270558    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/ca872639-3127-4885-ac9e-e92b0f9aa4dc-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:12.270615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.270568    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/ca872639-3127-4885-ac9e-e92b0f9aa4dc-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:12.684576 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.684542    5746 generic.go:356] "Generic (PLEG): container finished" podID="ca872639-3127-4885-ac9e-e92b0f9aa4dc" containerID="9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313" exitCode=0
Oct 30 18:57:12.684788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.684590    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"
Oct 30 18:57:12.684788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.684625    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65" event={"ID":"ca872639-3127-4885-ac9e-e92b0f9aa4dc","Type":"ContainerDied","Data":"9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313"}
Oct 30 18:57:12.684788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.684670    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65" event={"ID":"ca872639-3127-4885-ac9e-e92b0f9aa4dc","Type":"ContainerDied","Data":"c3bc31f4fd5af70e2f4d1acc5cfa323e983c503165ab1540478a8bc82e0ff9d3"}
Oct 30 18:57:12.684788 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.684695    5746 scope.go:122] "RemoveContainer" containerID="9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313"
Oct 30 18:57:12.693170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.693153    5746 scope.go:122] "RemoveContainer" containerID="9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313"
Oct 30 18:57:12.693424 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:57:12.693407    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313\": container with ID starting with 9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313 not found: ID does not exist" containerID="9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313"
Oct 30 18:57:12.693490 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.693437    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313"} err="failed to get container status \"9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313\": rpc error: code = NotFound desc = could not find container \"9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313\": container with ID starting with 9b8e4b35453b3a0c3e840fe5161fb5b8b158f3cf06fce8ce30fca78d27c28313 not found: ID does not exist"
Oct 30 18:57:12.737209 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.737171    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"]
Oct 30 18:57:12.748980 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:12.748944    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-6c8476f944-k4g65"]
Oct 30 18:57:12.790534 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:57:12.790500    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:57:14.009044 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:14.009014    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca872639-3127-4885-ac9e-e92b0f9aa4dc" path="/var/lib/kubelet/pods/ca872639-3127-4885-ac9e-e92b0f9aa4dc/volumes"
Oct 30 18:57:20.139968 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.139868    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:20.268503 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.268463    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"]
Oct 30 18:57:20.268685 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.268657    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj" podUID="68298f80-c1ef-47a7-9ed7-f149e025076e" containerName="ta-container" containerID="cri-o://7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec" gracePeriod=30
Oct 30 18:57:20.355632 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.355602    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:57:20.355835 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.355805    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="51bb24b3-472b-4a94-924f-1078a8c222e8" containerName="perses" containerID="cri-o://0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc" gracePeriod=30
Oct 30 18:57:20.640023 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.639999    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:20.642601 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.642581    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:20.643893 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.643873    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68298f80-c1ef-47a7-9ed7-f149e025076e-ta-internal\" (UniqueName: \"kubernetes.io/configmap/68298f80-c1ef-47a7-9ed7-f149e025076e-ta-internal\") pod \"68298f80-c1ef-47a7-9ed7-f149e025076e\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") "
Oct 30 18:57:20.643970 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.643953    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/68298f80-c1ef-47a7-9ed7-f149e025076e-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/68298f80-c1ef-47a7-9ed7-f149e025076e-data-science-collector-ta-server-cert\") pod \"68298f80-c1ef-47a7-9ed7-f149e025076e\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") "
Oct 30 18:57:20.644038 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.644023    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68298f80-c1ef-47a7-9ed7-f149e025076e-kube-api-access-js9gj\" (UniqueName: \"kubernetes.io/projected/68298f80-c1ef-47a7-9ed7-f149e025076e-kube-api-access-js9gj\") pod \"68298f80-c1ef-47a7-9ed7-f149e025076e\" (UID: \"68298f80-c1ef-47a7-9ed7-f149e025076e\") "
Oct 30 18:57:20.644283 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.644254    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68298f80-c1ef-47a7-9ed7-f149e025076e-ta-internal" pod "68298f80-c1ef-47a7-9ed7-f149e025076e" (UID: "68298f80-c1ef-47a7-9ed7-f149e025076e"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:20.646151 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.646129    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68298f80-c1ef-47a7-9ed7-f149e025076e-data-science-collector-ta-server-cert" pod "68298f80-c1ef-47a7-9ed7-f149e025076e" (UID: "68298f80-c1ef-47a7-9ed7-f149e025076e"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:20.646293 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.646272    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68298f80-c1ef-47a7-9ed7-f149e025076e-kube-api-access-js9gj" pod "68298f80-c1ef-47a7-9ed7-f149e025076e" (UID: "68298f80-c1ef-47a7-9ed7-f149e025076e"). InnerVolumeSpecName "kube-api-access-js9gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:20.715114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.715031    5746 generic.go:356] "Generic (PLEG): container finished" podID="51bb24b3-472b-4a94-924f-1078a8c222e8" containerID="0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc" exitCode=0
Oct 30 18:57:20.715114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.715074    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:20.715114 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.715098    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"51bb24b3-472b-4a94-924f-1078a8c222e8","Type":"ContainerDied","Data":"0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc"}
Oct 30 18:57:20.715380 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.715137    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"51bb24b3-472b-4a94-924f-1078a8c222e8","Type":"ContainerDied","Data":"f63a40722eacab93e2526ac87140e844b86871ab92f6d94176cc742a5b9876cf"}
Oct 30 18:57:20.715380 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.715162    5746 scope.go:122] "RemoveContainer" containerID="0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc"
Oct 30 18:57:20.716317 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.716297    5746 generic.go:356] "Generic (PLEG): container finished" podID="68298f80-c1ef-47a7-9ed7-f149e025076e" containerID="7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec" exitCode=0
Oct 30 18:57:20.716415 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.716334    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"
Oct 30 18:57:20.716415 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.716332    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj" event={"ID":"68298f80-c1ef-47a7-9ed7-f149e025076e","Type":"ContainerDied","Data":"7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec"}
Oct 30 18:57:20.716694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.716437    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj" event={"ID":"68298f80-c1ef-47a7-9ed7-f149e025076e","Type":"ContainerDied","Data":"596bb8879eb4222d8f1236fffc2ba21c5380036c0837db3305b2d1688ff37002"}
Oct 30 18:57:20.724820 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.724804    5746 scope.go:122] "RemoveContainer" containerID="0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc"
Oct 30 18:57:20.725118 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:57:20.725100    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc\": container with ID starting with 0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc not found: ID does not exist" containerID="0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc"
Oct 30 18:57:20.725185 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.725126    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc"} err="failed to get container status \"0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc\": rpc error: code = NotFound desc = could not find container \"0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc\": container with ID starting with 0053bc84351273d0ee54460f273aab677b0eb47dc1f41f4f01a9200e8a2889dc not found: ID does not exist"
Oct 30 18:57:20.725185 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.725143    5746 scope.go:122] "RemoveContainer" containerID="7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec"
Oct 30 18:57:20.732854 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.732834    5746 scope.go:122] "RemoveContainer" containerID="7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec"
Oct 30 18:57:20.733132 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:57:20.733114    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec\": container with ID starting with 7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec not found: ID does not exist" containerID="7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec"
Oct 30 18:57:20.733199 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.733138    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec"} err="failed to get container status \"7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec\": rpc error: code = NotFound desc = could not find container \"7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec\": container with ID starting with 7221a7818c861d766989878ca6e52a4bdf49d8bd7844693aee4b74b9410f95ec not found: ID does not exist"
Oct 30 18:57:20.744544 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.744525    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51bb24b3-472b-4a94-924f-1078a8c222e8-plugins\" (UniqueName: \"kubernetes.io/empty-dir/51bb24b3-472b-4a94-924f-1078a8c222e8-plugins\") pod \"51bb24b3-472b-4a94-924f-1078a8c222e8\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") "
Oct 30 18:57:20.744659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.744645    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"51bb24b3-472b-4a94-924f-1078a8c222e8\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") "
Oct 30 18:57:20.744727 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.744711    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51bb24b3-472b-4a94-924f-1078a8c222e8-kube-api-access-k2s95\" (UniqueName: \"kubernetes.io/projected/51bb24b3-472b-4a94-924f-1078a8c222e8-kube-api-access-k2s95\") pod \"51bb24b3-472b-4a94-924f-1078a8c222e8\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") "
Oct 30 18:57:20.744789 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.744770    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51bb24b3-472b-4a94-924f-1078a8c222e8-config\" (UniqueName: \"kubernetes.io/configmap/51bb24b3-472b-4a94-924f-1078a8c222e8-config\") pod \"51bb24b3-472b-4a94-924f-1078a8c222e8\" (UID: \"51bb24b3-472b-4a94-924f-1078a8c222e8\") "
Oct 30 18:57:20.745017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.744998    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js9gj\" (UniqueName: \"kubernetes.io/projected/68298f80-c1ef-47a7-9ed7-f149e025076e-kube-api-access-js9gj\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:20.745124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.745019    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/68298f80-c1ef-47a7-9ed7-f149e025076e-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:20.745124 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.745032    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/68298f80-c1ef-47a7-9ed7-f149e025076e-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:20.745245 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.745183    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51bb24b3-472b-4a94-924f-1078a8c222e8-config" pod "51bb24b3-472b-4a94-924f-1078a8c222e8" (UID: "51bb24b3-472b-4a94-924f-1078a8c222e8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:20.747107 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.747084    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51bb24b3-472b-4a94-924f-1078a8c222e8-kube-api-access-k2s95" pod "51bb24b3-472b-4a94-924f-1078a8c222e8" (UID: "51bb24b3-472b-4a94-924f-1078a8c222e8"). InnerVolumeSpecName "kube-api-access-k2s95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:20.747445 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.747429    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "51bb24b3-472b-4a94-924f-1078a8c222e8" (UID: "51bb24b3-472b-4a94-924f-1078a8c222e8"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:57:20.784019 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.783996    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"]
Oct 30 18:57:20.788337 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.788314    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-74bf949bc9-g2smj"]
Oct 30 18:57:20.845667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.845639    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:57:20.845667 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.845663    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2s95\" (UniqueName: \"kubernetes.io/projected/51bb24b3-472b-4a94-924f-1078a8c222e8-kube-api-access-k2s95\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:20.845837 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.845675    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/51bb24b3-472b-4a94-924f-1078a8c222e8-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:20.858603 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.858583    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:57:20.891259 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.891228    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51bb24b3-472b-4a94-924f-1078a8c222e8-plugins" pod "51bb24b3-472b-4a94-924f-1078a8c222e8" (UID: "51bb24b3-472b-4a94-924f-1078a8c222e8"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:20.946042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.946009    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/51bb24b3-472b-4a94-924f-1078a8c222e8-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:20.946042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:20.946038    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:57:21.057263 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:21.057233    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:57:21.063887 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:21.063857    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:57:22.007618 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:22.007584    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51bb24b3-472b-4a94-924f-1078a8c222e8" path="/var/lib/kubelet/pods/51bb24b3-472b-4a94-924f-1078a8c222e8/volumes"
Oct 30 18:57:22.008006 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:22.007967    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68298f80-c1ef-47a7-9ed7-f149e025076e" path="/var/lib/kubelet/pods/68298f80-c1ef-47a7-9ed7-f149e025076e/volumes"
Oct 30 18:57:30.140612 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:30.140576    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:40.140003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:40.139967    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:42.790165 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:57:42.790124    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:57:50.140472 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.140427    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:50.355240 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355203    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"]
Oct 30 18:57:50.355685 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355668    5746 cpu_manager.go:414] "containerMap: removing container" podUID="68298f80-c1ef-47a7-9ed7-f149e025076e" containerName="ta-container"
Oct 30 18:57:50.355777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355688    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68298f80-c1ef-47a7-9ed7-f149e025076e" containerName="ta-container"
Oct 30 18:57:50.355777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355707    5746 cpu_manager.go:414] "containerMap: removing container" podUID="ca872639-3127-4885-ac9e-e92b0f9aa4dc" containerName="ta-container"
Oct 30 18:57:50.355777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355715    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca872639-3127-4885-ac9e-e92b0f9aa4dc" containerName="ta-container"
Oct 30 18:57:50.355777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355740    5746 cpu_manager.go:414] "containerMap: removing container" podUID="51bb24b3-472b-4a94-924f-1078a8c222e8" containerName="perses"
Oct 30 18:57:50.355777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355749    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51bb24b3-472b-4a94-924f-1078a8c222e8" containerName="perses"
Oct 30 18:57:50.355963 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355807    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="68298f80-c1ef-47a7-9ed7-f149e025076e" containerName="ta-container"
Oct 30 18:57:50.355963 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355818    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca872639-3127-4885-ac9e-e92b0f9aa4dc" containerName="ta-container"
Oct 30 18:57:50.355963 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.355829    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="51bb24b3-472b-4a94-924f-1078a8c222e8" containerName="perses"
Oct 30 18:57:50.359034 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.359017    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"
Oct 30 18:57:50.382658 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.382630    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"]
Oct 30 18:57:50.393517 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.393459    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:57:50.498597 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.498562    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcr4t\" (UniqueName: \"kubernetes.io/projected/c6783180-922f-4e6f-a6de-09c86e5a63bf-kube-api-access-kcr4t\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc\" (UID: \"c6783180-922f-4e6f-a6de-09c86e5a63bf\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"
Oct 30 18:57:50.598948 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.598898    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcr4t\" (UniqueName: \"kubernetes.io/projected/c6783180-922f-4e6f-a6de-09c86e5a63bf-kube-api-access-kcr4t\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc\" (UID: \"c6783180-922f-4e6f-a6de-09c86e5a63bf\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"
Oct 30 18:57:50.621951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.621922    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcr4t\" (UniqueName: \"kubernetes.io/projected/c6783180-922f-4e6f-a6de-09c86e5a63bf-kube-api-access-kcr4t\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc\" (UID: \"c6783180-922f-4e6f-a6de-09c86e5a63bf\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"
Oct 30 18:57:50.670897 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.670839    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"
Oct 30 18:57:50.962579 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:50.962510    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"]
Oct 30 18:57:50.966487 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:57:50.966433    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6783180_922f_4e6f_a6de_09c86e5a63bf.slice/crio-0a30beeeb7eb052f53f7c970897fc159f19a6fd669e93b8952842c835f6cd0c0 WatchSource:0}: Error finding container 0a30beeeb7eb052f53f7c970897fc159f19a6fd669e93b8952842c835f6cd0c0: Status 404 returned error can't find the container with id 0a30beeeb7eb052f53f7c970897fc159f19a6fd669e93b8952842c835f6cd0c0
Oct 30 18:57:51.365436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.365404    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:57:51.368841 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.368819    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.392110 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.392082    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:57:51.393170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.393138    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:57:51.466082 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.466055    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"]
Oct 30 18:57:51.469224 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.469205    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.484111 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.484090    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:57:51.488924 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.488883    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"]
Oct 30 18:57:51.506376 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.506356    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37706318-56e5-4d99-9fdd-8182f964c144-config\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.506489 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.506390    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx4tg\" (UniqueName: \"kubernetes.io/projected/37706318-56e5-4d99-9fdd-8182f964c144-kube-api-access-wx4tg\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.506489 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.506414    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/37706318-56e5-4d99-9fdd-8182f964c144-plugins\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.519218 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.519199    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:57:51.519311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.519261    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-hg9vh\""
Oct 30 18:57:51.607863 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.607829    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37706318-56e5-4d99-9fdd-8182f964c144-config\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.608056 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.607876    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72cm4\" (UniqueName: \"kubernetes.io/projected/4382268a-e56d-4a22-bfe2-e2779178e498-kube-api-access-72cm4\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.608056 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.607895    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx4tg\" (UniqueName: \"kubernetes.io/projected/37706318-56e5-4d99-9fdd-8182f964c144-kube-api-access-wx4tg\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.608056 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.608024    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/37706318-56e5-4d99-9fdd-8182f964c144-plugins\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.608163 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.608068    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/4382268a-e56d-4a22-bfe2-e2779178e498-ta-internal\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.608163 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.608085    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/4382268a-e56d-4a22-bfe2-e2779178e498-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.608418 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.608399    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/37706318-56e5-4d99-9fdd-8182f964c144-plugins\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.608586 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.608570    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37706318-56e5-4d99-9fdd-8182f964c144-config\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.632436 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.632367    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx4tg\" (UniqueName: \"kubernetes.io/projected/37706318-56e5-4d99-9fdd-8182f964c144-kube-api-access-wx4tg\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:51.709233 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.709200    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72cm4\" (UniqueName: \"kubernetes.io/projected/4382268a-e56d-4a22-bfe2-e2779178e498-kube-api-access-72cm4\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.709408 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.709275    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/4382268a-e56d-4a22-bfe2-e2779178e498-ta-internal\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.709408 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.709294    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/4382268a-e56d-4a22-bfe2-e2779178e498-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.710003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.709947    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/4382268a-e56d-4a22-bfe2-e2779178e498-ta-internal\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.711860 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.711836    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/4382268a-e56d-4a22-bfe2-e2779178e498-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.734878 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.734849    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72cm4\" (UniqueName: \"kubernetes.io/projected/4382268a-e56d-4a22-bfe2-e2779178e498-kube-api-access-72cm4\") pod \"data-science-collector-targetallocator-544874d5-w5btf\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.780468 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.780431    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:51.840938 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.838997    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc" event={"ID":"c6783180-922f-4e6f-a6de-09c86e5a63bf","Type":"ContainerStarted","Data":"a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4"}
Oct 30 18:57:51.840938 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.839043    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc" event={"ID":"c6783180-922f-4e6f-a6de-09c86e5a63bf","Type":"ContainerStarted","Data":"0a30beeeb7eb052f53f7c970897fc159f19a6fd669e93b8952842c835f6cd0c0"}
Oct 30 18:57:51.877352 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:51.877303    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc" podStartSLOduration=1.877284347 podStartE2EDuration="1.877284347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:51.876869803 +0000 UTC m=+3642.043687962" watchObservedRunningTime="2026-10-30 18:57:51.877284347 +0000 UTC m=+3642.044102477"
Oct 30 18:57:52.076697 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:52.076622    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"]
Oct 30 18:57:52.079856 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:57:52.079830    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4382268a_e56d_4a22_bfe2_e2779178e498.slice/crio-94e8893bb1783c5c62e24704f4ace2b5993779821a3decfe8b6f2522d6ea0ef7 WatchSource:0}: Error finding container 94e8893bb1783c5c62e24704f4ace2b5993779821a3decfe8b6f2522d6ea0ef7: Status 404 returned error can't find the container with id 94e8893bb1783c5c62e24704f4ace2b5993779821a3decfe8b6f2522d6ea0ef7
Oct 30 18:57:52.081969 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:52.081936    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:57:52.082054 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:52.081999    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:57:52.844640 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:52.844604    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf" event={"ID":"4382268a-e56d-4a22-bfe2-e2779178e498","Type":"ContainerStarted","Data":"5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39"}
Oct 30 18:57:52.845098 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:52.844656    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:52.845098 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:52.844673    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf" event={"ID":"4382268a-e56d-4a22-bfe2-e2779178e498","Type":"ContainerStarted","Data":"94e8893bb1783c5c62e24704f4ace2b5993779821a3decfe8b6f2522d6ea0ef7"}
Oct 30 18:57:52.846599 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:52.846579    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:57:52.885632 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:52.885575    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf" podStartSLOduration=1.885557962 podStartE2EDuration="1.885557962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:52.884930357 +0000 UTC m=+3643.051748481" watchObservedRunningTime="2026-10-30 18:57:52.885557962 +0000 UTC m=+3643.052376091"
Oct 30 18:57:56.047116 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.047083    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.075640 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.075612    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.147672 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.147634    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.147867 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.147794    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.162105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.162083    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.268343 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.268315    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.298292 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.298227    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.479637 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.479597    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.775596 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.775560    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:57:56.779091 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:57:56.779059    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37706318_56e5_4d99_9fdd_8182f964c144.slice/crio-851b3394efb0df57e9b42eff8958ea19a6c92ecf9fa6c51a4ae33742bd0339df WatchSource:0}: Error finding container 851b3394efb0df57e9b42eff8958ea19a6c92ecf9fa6c51a4ae33742bd0339df: Status 404 returned error can't find the container with id 851b3394efb0df57e9b42eff8958ea19a6c92ecf9fa6c51a4ae33742bd0339df
Oct 30 18:57:56.863524 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.863494    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"37706318-56e5-4d99-9fdd-8182f964c144","Type":"ContainerStarted","Data":"5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73"}
Oct 30 18:57:56.863684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.863534    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"37706318-56e5-4d99-9fdd-8182f964c144","Type":"ContainerStarted","Data":"851b3394efb0df57e9b42eff8958ea19a6c92ecf9fa6c51a4ae33742bd0339df"}
Oct 30 18:57:56.863684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.863577    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:57:56.905407 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:56.905356    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=5.905340337 podStartE2EDuration="5.905340337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:56.904597614 +0000 UTC m=+3647.071415761" watchObservedRunningTime="2026-10-30 18:57:56.905340337 +0000 UTC m=+3647.072158464"
Oct 30 18:57:59.193721 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.193689    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"]
Oct 30 18:57:59.194153 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.193853    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc" podUID="c6783180-922f-4e6f-a6de-09c86e5a63bf" containerName="thanos-querier" containerID="cri-o://a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4" gracePeriod=30
Oct 30 18:57:59.448320 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.448264    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"
Oct 30 18:57:59.475733 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.475705    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6783180-922f-4e6f-a6de-09c86e5a63bf-kube-api-access-kcr4t\" (UniqueName: \"kubernetes.io/projected/c6783180-922f-4e6f-a6de-09c86e5a63bf-kube-api-access-kcr4t\") pod \"c6783180-922f-4e6f-a6de-09c86e5a63bf\" (UID: \"c6783180-922f-4e6f-a6de-09c86e5a63bf\") "
Oct 30 18:57:59.477992 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.477967    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6783180-922f-4e6f-a6de-09c86e5a63bf-kube-api-access-kcr4t" pod "c6783180-922f-4e6f-a6de-09c86e5a63bf" (UID: "c6783180-922f-4e6f-a6de-09c86e5a63bf"). InnerVolumeSpecName "kube-api-access-kcr4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:59.576334 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.576301    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcr4t\" (UniqueName: \"kubernetes.io/projected/c6783180-922f-4e6f-a6de-09c86e5a63bf-kube-api-access-kcr4t\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:57:59.877662 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.877631    5746 generic.go:356] "Generic (PLEG): container finished" podID="c6783180-922f-4e6f-a6de-09c86e5a63bf" containerID="a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4" exitCode=0
Oct 30 18:57:59.877829 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.877676    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"
Oct 30 18:57:59.877829 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.877719    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc" event={"ID":"c6783180-922f-4e6f-a6de-09c86e5a63bf","Type":"ContainerDied","Data":"a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4"}
Oct 30 18:57:59.877829 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.877756    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc" event={"ID":"c6783180-922f-4e6f-a6de-09c86e5a63bf","Type":"ContainerDied","Data":"0a30beeeb7eb052f53f7c970897fc159f19a6fd669e93b8952842c835f6cd0c0"}
Oct 30 18:57:59.877829 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.877773    5746 scope.go:122] "RemoveContainer" containerID="a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4"
Oct 30 18:57:59.892170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.892152    5746 scope.go:122] "RemoveContainer" containerID="a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4"
Oct 30 18:57:59.892422 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:57:59.892402    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4\": container with ID starting with a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4 not found: ID does not exist" containerID="a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4"
Oct 30 18:57:59.892470 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.892431    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4"} err="failed to get container status \"a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4\": rpc error: code = NotFound desc = could not find container \"a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4\": container with ID starting with a0c6d4230f83aacb2e25bd36b24d456f82665e6c84615f31c5c5310b54a13ea4 not found: ID does not exist"
Oct 30 18:57:59.928875 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.928837    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"]
Oct 30 18:57:59.932593 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:57:59.932569    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-tx9qc"]
Oct 30 18:58:00.008152 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:00.008122    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6783180-922f-4e6f-a6de-09c86e5a63bf" path="/var/lib/kubelet/pods/c6783180-922f-4e6f-a6de-09c86e5a63bf/volumes"
Oct 30 18:58:00.140445 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:00.140355    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:07.870091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:07.870065    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:58:10.139644 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:10.139606    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:12.791254 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:58:12.791209    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:58:20.140073 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:20.140030    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:22.889319 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.889285    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"]
Oct 30 18:58:22.889719 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.889549    5746 cpu_manager.go:414] "containerMap: removing container" podUID="c6783180-922f-4e6f-a6de-09c86e5a63bf" containerName="thanos-querier"
Oct 30 18:58:22.889719 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.889559    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6783180-922f-4e6f-a6de-09c86e5a63bf" containerName="thanos-querier"
Oct 30 18:58:22.889719 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.889643    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6783180-922f-4e6f-a6de-09c86e5a63bf" containerName="thanos-querier"
Oct 30 18:58:22.893161 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.893138    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:22.919786 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.919757    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"]
Oct 30 18:58:22.971248 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.971220    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f634f01-e0eb-4285-8624-c3c6d6e54e11-ta-internal\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:22.971248 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.971251    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f634f01-e0eb-4285-8624-c3c6d6e54e11-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:22.971441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:22.971373    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-762f7\" (UniqueName: \"kubernetes.io/projected/3f634f01-e0eb-4285-8624-c3c6d6e54e11-kube-api-access-762f7\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.071778 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.071736    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-762f7\" (UniqueName: \"kubernetes.io/projected/3f634f01-e0eb-4285-8624-c3c6d6e54e11-kube-api-access-762f7\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.072025 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.071812    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f634f01-e0eb-4285-8624-c3c6d6e54e11-ta-internal\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.072025 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.071841    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f634f01-e0eb-4285-8624-c3c6d6e54e11-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.072447 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.072426    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f634f01-e0eb-4285-8624-c3c6d6e54e11-ta-internal\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.074607 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.074571    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f634f01-e0eb-4285-8624-c3c6d6e54e11-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.094651 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.094625    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-762f7\" (UniqueName: \"kubernetes.io/projected/3f634f01-e0eb-4285-8624-c3c6d6e54e11-kube-api-access-762f7\") pod \"data-science-collector-targetallocator-7fdd66d468-9gg9l\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.204706 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.204620    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.499809 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.499734    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"]
Oct 30 18:58:23.503296 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:58:23.503265    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f634f01_e0eb_4285_8624_c3c6d6e54e11.slice/crio-138821ad8e7a4452ff61985cdd9c501e9ef45e6841496002f7437da2783f6100 WatchSource:0}: Error finding container 138821ad8e7a4452ff61985cdd9c501e9ef45e6841496002f7437da2783f6100: Status 404 returned error can't find the container with id 138821ad8e7a4452ff61985cdd9c501e9ef45e6841496002f7437da2783f6100
Oct 30 18:58:23.505464 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.505420    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:58:23.505560 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.505500    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:58:23.974776 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.974739    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l" event={"ID":"3f634f01-e0eb-4285-8624-c3c6d6e54e11","Type":"ContainerStarted","Data":"2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180"}
Oct 30 18:58:23.975253 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.974788    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l" event={"ID":"3f634f01-e0eb-4285-8624-c3c6d6e54e11","Type":"ContainerStarted","Data":"138821ad8e7a4452ff61985cdd9c501e9ef45e6841496002f7437da2783f6100"}
Oct 30 18:58:23.975253 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.974946    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:23.976990 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:23.976973    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:24.015961 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.015878    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l" podStartSLOduration=2.015859883 podStartE2EDuration="2.015859883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:24.015455015 +0000 UTC m=+3674.182273183" watchObservedRunningTime="2026-10-30 18:58:24.015859883 +0000 UTC m=+3674.182678010"
Oct 30 18:58:24.089720 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.089327    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"]
Oct 30 18:58:24.089720 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.089537    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf" podUID="4382268a-e56d-4a22-bfe2-e2779178e498" containerName="ta-container" containerID="cri-o://5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39" gracePeriod=30
Oct 30 18:58:24.352953 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.352928    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:58:24.390660 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.390628    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4382268a-e56d-4a22-bfe2-e2779178e498-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/4382268a-e56d-4a22-bfe2-e2779178e498-data-science-collector-ta-server-cert\") pod \"4382268a-e56d-4a22-bfe2-e2779178e498\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") "
Oct 30 18:58:24.390828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.390701    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4382268a-e56d-4a22-bfe2-e2779178e498-kube-api-access-72cm4\" (UniqueName: \"kubernetes.io/projected/4382268a-e56d-4a22-bfe2-e2779178e498-kube-api-access-72cm4\") pod \"4382268a-e56d-4a22-bfe2-e2779178e498\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") "
Oct 30 18:58:24.390828 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.390725    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4382268a-e56d-4a22-bfe2-e2779178e498-ta-internal\" (UniqueName: \"kubernetes.io/configmap/4382268a-e56d-4a22-bfe2-e2779178e498-ta-internal\") pod \"4382268a-e56d-4a22-bfe2-e2779178e498\" (UID: \"4382268a-e56d-4a22-bfe2-e2779178e498\") "
Oct 30 18:58:24.391134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.391108    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4382268a-e56d-4a22-bfe2-e2779178e498-ta-internal" pod "4382268a-e56d-4a22-bfe2-e2779178e498" (UID: "4382268a-e56d-4a22-bfe2-e2779178e498"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:24.393055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.393032    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4382268a-e56d-4a22-bfe2-e2779178e498-data-science-collector-ta-server-cert" pod "4382268a-e56d-4a22-bfe2-e2779178e498" (UID: "4382268a-e56d-4a22-bfe2-e2779178e498"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:58:24.393142 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.393030    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4382268a-e56d-4a22-bfe2-e2779178e498-kube-api-access-72cm4" pod "4382268a-e56d-4a22-bfe2-e2779178e498" (UID: "4382268a-e56d-4a22-bfe2-e2779178e498"). InnerVolumeSpecName "kube-api-access-72cm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:24.491243 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.491208    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72cm4\" (UniqueName: \"kubernetes.io/projected/4382268a-e56d-4a22-bfe2-e2779178e498-kube-api-access-72cm4\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:24.491243 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.491242    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/4382268a-e56d-4a22-bfe2-e2779178e498-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:24.491456 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.491253    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/4382268a-e56d-4a22-bfe2-e2779178e498-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:24.981212 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.981175    5746 generic.go:356] "Generic (PLEG): container finished" podID="4382268a-e56d-4a22-bfe2-e2779178e498" containerID="5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39" exitCode=0
Oct 30 18:58:24.981709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.981238    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"
Oct 30 18:58:24.981709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.981257    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf" event={"ID":"4382268a-e56d-4a22-bfe2-e2779178e498","Type":"ContainerDied","Data":"5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39"}
Oct 30 18:58:24.981709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.981302    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf" event={"ID":"4382268a-e56d-4a22-bfe2-e2779178e498","Type":"ContainerDied","Data":"94e8893bb1783c5c62e24704f4ace2b5993779821a3decfe8b6f2522d6ea0ef7"}
Oct 30 18:58:24.981709 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.981319    5746 scope.go:122] "RemoveContainer" containerID="5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39"
Oct 30 18:58:24.989611 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.989596    5746 scope.go:122] "RemoveContainer" containerID="5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39"
Oct 30 18:58:24.989895 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:58:24.989877    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39\": container with ID starting with 5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39 not found: ID does not exist" containerID="5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39"
Oct 30 18:58:24.989993 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:24.989902    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39"} err="failed to get container status \"5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39\": rpc error: code = NotFound desc = could not find container \"5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39\": container with ID starting with 5301e46b654d827d6dde6a0bac48a8d7f2b61f8e3d9d4a1a674f5cfe850a6e39 not found: ID does not exist"
Oct 30 18:58:25.030685 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:25.030651    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"]
Oct 30 18:58:25.042748 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:25.042722    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-544874d5-w5btf"]
Oct 30 18:58:26.007507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:26.007476    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4382268a-e56d-4a22-bfe2-e2779178e498" path="/var/lib/kubelet/pods/4382268a-e56d-4a22-bfe2-e2779178e498/volumes"
Oct 30 18:58:30.140858 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:30.140824    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:39.663665 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.663632    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"]
Oct 30 18:58:39.664091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.663929    5746 cpu_manager.go:414] "containerMap: removing container" podUID="4382268a-e56d-4a22-bfe2-e2779178e498" containerName="ta-container"
Oct 30 18:58:39.664091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.663941    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4382268a-e56d-4a22-bfe2-e2779178e498" containerName="ta-container"
Oct 30 18:58:39.664091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.664016    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="4382268a-e56d-4a22-bfe2-e2779178e498" containerName="ta-container"
Oct 30 18:58:39.668193 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.668171    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.700304 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.700279    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"]
Oct 30 18:58:39.707958 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.707932    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/20b9ecc8-f39f-4af5-9bc8-da903375047b-ta-internal\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.708089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.707992    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/20b9ecc8-f39f-4af5-9bc8-da903375047b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.708089 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.708063    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fsnq\" (UniqueName: \"kubernetes.io/projected/20b9ecc8-f39f-4af5-9bc8-da903375047b-kube-api-access-4fsnq\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.808768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.808737    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/20b9ecc8-f39f-4af5-9bc8-da903375047b-ta-internal\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.808972 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.808788    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/20b9ecc8-f39f-4af5-9bc8-da903375047b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.808972 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.808810    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fsnq\" (UniqueName: \"kubernetes.io/projected/20b9ecc8-f39f-4af5-9bc8-da903375047b-kube-api-access-4fsnq\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.809405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.809380    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/20b9ecc8-f39f-4af5-9bc8-da903375047b-ta-internal\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.811507 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.811478    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/20b9ecc8-f39f-4af5-9bc8-da903375047b-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.833520 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.833496    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fsnq\" (UniqueName: \"kubernetes.io/projected/20b9ecc8-f39f-4af5-9bc8-da903375047b-kube-api-access-4fsnq\") pod \"data-science-collector-targetallocator-ddf65cf84-rktps\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:39.979188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:39.979106    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:40.140302 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:40.140267    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:40.276525 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:40.276500    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"]
Oct 30 18:58:40.278143 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:58:40.278117    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20b9ecc8_f39f_4af5_9bc8_da903375047b.slice/crio-8052e8b547baaf31e934c892c4af8087929d55dadf41c0bbd4d01fac3fc27aa3 WatchSource:0}: Error finding container 8052e8b547baaf31e934c892c4af8087929d55dadf41c0bbd4d01fac3fc27aa3: Status 404 returned error can't find the container with id 8052e8b547baaf31e934c892c4af8087929d55dadf41c0bbd4d01fac3fc27aa3
Oct 30 18:58:40.280949 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:40.280883    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:58:40.281035 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:40.280988    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:58:41.043754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.043713    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps" event={"ID":"20b9ecc8-f39f-4af5-9bc8-da903375047b","Type":"ContainerStarted","Data":"d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c"}
Oct 30 18:58:41.043754 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.043760    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps" event={"ID":"20b9ecc8-f39f-4af5-9bc8-da903375047b","Type":"ContainerStarted","Data":"8052e8b547baaf31e934c892c4af8087929d55dadf41c0bbd4d01fac3fc27aa3"}
Oct 30 18:58:41.044314 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.043865    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:41.045749 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.045729    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:41.088664 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.088619    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps" podStartSLOduration=2.0886069210000002 podStartE2EDuration="2.088606921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:41.087973741 +0000 UTC m=+3691.254791889" watchObservedRunningTime="2026-10-30 18:58:41.088606921 +0000 UTC m=+3691.255425047"
Oct 30 18:58:41.163180 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.163152    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"]
Oct 30 18:58:41.163427 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.163339    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l" podUID="3f634f01-e0eb-4285-8624-c3c6d6e54e11" containerName="ta-container" containerID="cri-o://2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180" gracePeriod=30
Oct 30 18:58:41.434290 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.434265    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:41.523493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.523459    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f634f01-e0eb-4285-8624-c3c6d6e54e11-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f634f01-e0eb-4285-8624-c3c6d6e54e11-data-science-collector-ta-server-cert\") pod \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") "
Oct 30 18:58:41.523493 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.523501    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f634f01-e0eb-4285-8624-c3c6d6e54e11-kube-api-access-762f7\" (UniqueName: \"kubernetes.io/projected/3f634f01-e0eb-4285-8624-c3c6d6e54e11-kube-api-access-762f7\") pod \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") "
Oct 30 18:58:41.523753 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.523560    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f634f01-e0eb-4285-8624-c3c6d6e54e11-ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f634f01-e0eb-4285-8624-c3c6d6e54e11-ta-internal\") pod \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\" (UID: \"3f634f01-e0eb-4285-8624-c3c6d6e54e11\") "
Oct 30 18:58:41.523864 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.523843    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f634f01-e0eb-4285-8624-c3c6d6e54e11-ta-internal" pod "3f634f01-e0eb-4285-8624-c3c6d6e54e11" (UID: "3f634f01-e0eb-4285-8624-c3c6d6e54e11"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:41.525756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.525733    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f634f01-e0eb-4285-8624-c3c6d6e54e11-data-science-collector-ta-server-cert" pod "3f634f01-e0eb-4285-8624-c3c6d6e54e11" (UID: "3f634f01-e0eb-4285-8624-c3c6d6e54e11"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:58:41.525836 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.525803    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f634f01-e0eb-4285-8624-c3c6d6e54e11-kube-api-access-762f7" pod "3f634f01-e0eb-4285-8624-c3c6d6e54e11" (UID: "3f634f01-e0eb-4285-8624-c3c6d6e54e11"). InnerVolumeSpecName "kube-api-access-762f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:41.624273 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.624224    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f634f01-e0eb-4285-8624-c3c6d6e54e11-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:41.624273 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.624271    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-762f7\" (UniqueName: \"kubernetes.io/projected/3f634f01-e0eb-4285-8624-c3c6d6e54e11-kube-api-access-762f7\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:41.624475 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:41.624292    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f634f01-e0eb-4285-8624-c3c6d6e54e11-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:42.048896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.048862    5746 generic.go:356] "Generic (PLEG): container finished" podID="3f634f01-e0eb-4285-8624-c3c6d6e54e11" containerID="2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180" exitCode=0
Oct 30 18:58:42.049384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.048951    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l" event={"ID":"3f634f01-e0eb-4285-8624-c3c6d6e54e11","Type":"ContainerDied","Data":"2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180"}
Oct 30 18:58:42.049384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.049018    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l" event={"ID":"3f634f01-e0eb-4285-8624-c3c6d6e54e11","Type":"ContainerDied","Data":"138821ad8e7a4452ff61985cdd9c501e9ef45e6841496002f7437da2783f6100"}
Oct 30 18:58:42.049384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.049045    5746 scope.go:122] "RemoveContainer" containerID="2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180"
Oct 30 18:58:42.049384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.048972    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"
Oct 30 18:58:42.058852 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.058833    5746 scope.go:122] "RemoveContainer" containerID="2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180"
Oct 30 18:58:42.059245 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:58:42.059225    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180\": container with ID starting with 2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180 not found: ID does not exist" containerID="2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180"
Oct 30 18:58:42.059325 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.059251    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180"} err="failed to get container status \"2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180\": rpc error: code = NotFound desc = could not find container \"2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180\": container with ID starting with 2c8d4ef820a099b689737c959babab77026ff81178e11673ecc54019c1f53180 not found: ID does not exist"
Oct 30 18:58:42.104512 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.104485    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"]
Oct 30 18:58:42.108890 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:42.108871    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-7fdd66d468-9gg9l"]
Oct 30 18:58:42.789871 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:58:42.789840    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:58:44.007706 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:44.007675    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f634f01-e0eb-4285-8624-c3c6d6e54e11" path="/var/lib/kubelet/pods/3f634f01-e0eb-4285-8624-c3c6d6e54e11/volumes"
Oct 30 18:58:49.373254 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.373221    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"]
Oct 30 18:58:49.373737 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.373413    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps" podUID="20b9ecc8-f39f-4af5-9bc8-da903375047b" containerName="ta-container" containerID="cri-o://d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c" gracePeriod=30
Oct 30 18:58:49.496615 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.496587    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:58:49.496882 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.496837    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="37706318-56e5-4d99-9fdd-8182f964c144" containerName="perses" containerID="cri-o://5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73" gracePeriod=30
Oct 30 18:58:49.646189 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.646163    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:49.686303 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.686278    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20b9ecc8-f39f-4af5-9bc8-da903375047b-ta-internal\" (UniqueName: \"kubernetes.io/configmap/20b9ecc8-f39f-4af5-9bc8-da903375047b-ta-internal\") pod \"20b9ecc8-f39f-4af5-9bc8-da903375047b\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") "
Oct 30 18:58:49.686435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.686312    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20b9ecc8-f39f-4af5-9bc8-da903375047b-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/20b9ecc8-f39f-4af5-9bc8-da903375047b-data-science-collector-ta-server-cert\") pod \"20b9ecc8-f39f-4af5-9bc8-da903375047b\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") "
Oct 30 18:58:49.686435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.686361    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20b9ecc8-f39f-4af5-9bc8-da903375047b-kube-api-access-4fsnq\" (UniqueName: \"kubernetes.io/projected/20b9ecc8-f39f-4af5-9bc8-da903375047b-kube-api-access-4fsnq\") pod \"20b9ecc8-f39f-4af5-9bc8-da903375047b\" (UID: \"20b9ecc8-f39f-4af5-9bc8-da903375047b\") "
Oct 30 18:58:49.686757 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.686724    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20b9ecc8-f39f-4af5-9bc8-da903375047b-ta-internal" pod "20b9ecc8-f39f-4af5-9bc8-da903375047b" (UID: "20b9ecc8-f39f-4af5-9bc8-da903375047b"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:49.688578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.688545    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20b9ecc8-f39f-4af5-9bc8-da903375047b-data-science-collector-ta-server-cert" pod "20b9ecc8-f39f-4af5-9bc8-da903375047b" (UID: "20b9ecc8-f39f-4af5-9bc8-da903375047b"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:58:49.688682 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.688617    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20b9ecc8-f39f-4af5-9bc8-da903375047b-kube-api-access-4fsnq" pod "20b9ecc8-f39f-4af5-9bc8-da903375047b" (UID: "20b9ecc8-f39f-4af5-9bc8-da903375047b"). InnerVolumeSpecName "kube-api-access-4fsnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:49.776638 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.776610    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:58:49.787659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.787638    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fsnq\" (UniqueName: \"kubernetes.io/projected/20b9ecc8-f39f-4af5-9bc8-da903375047b-kube-api-access-4fsnq\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:49.787659 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.787661    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/20b9ecc8-f39f-4af5-9bc8-da903375047b-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:49.787797 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.787673    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/20b9ecc8-f39f-4af5-9bc8-da903375047b-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:49.888474 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.888403    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"37706318-56e5-4d99-9fdd-8182f964c144\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") "
Oct 30 18:58:49.888474 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.888448    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37706318-56e5-4d99-9fdd-8182f964c144-config\" (UniqueName: \"kubernetes.io/configmap/37706318-56e5-4d99-9fdd-8182f964c144-config\") pod \"37706318-56e5-4d99-9fdd-8182f964c144\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") "
Oct 30 18:58:49.888474 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.888473    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37706318-56e5-4d99-9fdd-8182f964c144-kube-api-access-wx4tg\" (UniqueName: \"kubernetes.io/projected/37706318-56e5-4d99-9fdd-8182f964c144-kube-api-access-wx4tg\") pod \"37706318-56e5-4d99-9fdd-8182f964c144\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") "
Oct 30 18:58:49.888728 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.888530    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37706318-56e5-4d99-9fdd-8182f964c144-plugins\" (UniqueName: \"kubernetes.io/empty-dir/37706318-56e5-4d99-9fdd-8182f964c144-plugins\") pod \"37706318-56e5-4d99-9fdd-8182f964c144\" (UID: \"37706318-56e5-4d99-9fdd-8182f964c144\") "
Oct 30 18:58:49.888951 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.888890    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37706318-56e5-4d99-9fdd-8182f964c144-config" pod "37706318-56e5-4d99-9fdd-8182f964c144" (UID: "37706318-56e5-4d99-9fdd-8182f964c144"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:49.890818 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.890794    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37706318-56e5-4d99-9fdd-8182f964c144-kube-api-access-wx4tg" pod "37706318-56e5-4d99-9fdd-8182f964c144" (UID: "37706318-56e5-4d99-9fdd-8182f964c144"). InnerVolumeSpecName "kube-api-access-wx4tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:49.891354 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.891331    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "37706318-56e5-4d99-9fdd-8182f964c144" (UID: "37706318-56e5-4d99-9fdd-8182f964c144"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:58:49.989247 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.989216    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 18:58:49.989247 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.989242    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/37706318-56e5-4d99-9fdd-8182f964c144-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:49.989247 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:49.989254    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx4tg\" (UniqueName: \"kubernetes.io/projected/37706318-56e5-4d99-9fdd-8182f964c144-kube-api-access-wx4tg\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:50.001786 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.001764    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 18:58:50.036295 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.036266    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37706318-56e5-4d99-9fdd-8182f964c144-plugins" pod "37706318-56e5-4d99-9fdd-8182f964c144" (UID: "37706318-56e5-4d99-9fdd-8182f964c144"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:50.084720 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.084691    5746 generic.go:356] "Generic (PLEG): container finished" podID="20b9ecc8-f39f-4af5-9bc8-da903375047b" containerID="d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c" exitCode=0
Oct 30 18:58:50.084865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.084750    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"
Oct 30 18:58:50.084865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.084779    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps" event={"ID":"20b9ecc8-f39f-4af5-9bc8-da903375047b","Type":"ContainerDied","Data":"d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c"}
Oct 30 18:58:50.084865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.084824    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps" event={"ID":"20b9ecc8-f39f-4af5-9bc8-da903375047b","Type":"ContainerDied","Data":"8052e8b547baaf31e934c892c4af8087929d55dadf41c0bbd4d01fac3fc27aa3"}
Oct 30 18:58:50.084865 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.084846    5746 scope.go:122] "RemoveContainer" containerID="d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c"
Oct 30 18:58:50.086284 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.086252    5746 generic.go:356] "Generic (PLEG): container finished" podID="37706318-56e5-4d99-9fdd-8182f964c144" containerID="5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73" exitCode=0
Oct 30 18:58:50.086405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.086284    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"37706318-56e5-4d99-9fdd-8182f964c144","Type":"ContainerDied","Data":"5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73"}
Oct 30 18:58:50.086405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.086308    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:58:50.086405 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.086337    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"37706318-56e5-4d99-9fdd-8182f964c144","Type":"ContainerDied","Data":"851b3394efb0df57e9b42eff8958ea19a6c92ecf9fa6c51a4ae33742bd0339df"}
Oct 30 18:58:50.089603 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.089577    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 18:58:50.089603 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.089607    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/37706318-56e5-4d99-9fdd-8182f964c144-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:58:50.093589 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.093556    5746 scope.go:122] "RemoveContainer" containerID="d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c"
Oct 30 18:58:50.093872 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:58:50.093853    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c\": container with ID starting with d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c not found: ID does not exist" containerID="d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c"
Oct 30 18:58:50.093975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.093878    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c"} err="failed to get container status \"d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c\": rpc error: code = NotFound desc = could not find container \"d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c\": container with ID starting with d124db70e5848a4361cf179f55f3a2fad36b1c8be4089cdece1ab3440989f15c not found: ID does not exist"
Oct 30 18:58:50.093975 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.093896    5746 scope.go:122] "RemoveContainer" containerID="5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73"
Oct 30 18:58:50.101534 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.101517    5746 scope.go:122] "RemoveContainer" containerID="5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73"
Oct 30 18:58:50.101805 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:58:50.101786    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73\": container with ID starting with 5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73 not found: ID does not exist" containerID="5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73"
Oct 30 18:58:50.101871 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.101814    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73"} err="failed to get container status \"5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73\": rpc error: code = NotFound desc = could not find container \"5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73\": container with ID starting with 5b24196bcf2aefd54851cee6ad7adc77e6de8e59764de6f91cfc019dc31dff73 not found: ID does not exist"
Oct 30 18:58:50.139078 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.139026    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:58:50.141027 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.141004    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:50.152324 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.152303    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:58:50.180275 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.180251    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"]
Oct 30 18:58:50.185477 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:50.185459    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-ddf65cf84-rktps"]
Oct 30 18:58:52.007889 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:52.007859    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20b9ecc8-f39f-4af5-9bc8-da903375047b" path="/var/lib/kubelet/pods/20b9ecc8-f39f-4af5-9bc8-da903375047b/volumes"
Oct 30 18:58:52.008299 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:58:52.008265    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37706318-56e5-4d99-9fdd-8182f964c144" path="/var/lib/kubelet/pods/37706318-56e5-4d99-9fdd-8182f964c144/volumes"
Oct 30 18:59:00.141010 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.140967    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:59:00.175868 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.175838    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"]
Oct 30 18:59:00.176243 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176225    5746 cpu_manager.go:414] "containerMap: removing container" podUID="3f634f01-e0eb-4285-8624-c3c6d6e54e11" containerName="ta-container"
Oct 30 18:59:00.176311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176245    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f634f01-e0eb-4285-8624-c3c6d6e54e11" containerName="ta-container"
Oct 30 18:59:00.176311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176257    5746 cpu_manager.go:414] "containerMap: removing container" podUID="37706318-56e5-4d99-9fdd-8182f964c144" containerName="perses"
Oct 30 18:59:00.176311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176262    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37706318-56e5-4d99-9fdd-8182f964c144" containerName="perses"
Oct 30 18:59:00.176311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176271    5746 cpu_manager.go:414] "containerMap: removing container" podUID="20b9ecc8-f39f-4af5-9bc8-da903375047b" containerName="ta-container"
Oct 30 18:59:00.176311 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176276    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20b9ecc8-f39f-4af5-9bc8-da903375047b" containerName="ta-container"
Oct 30 18:59:00.176573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176329    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="37706318-56e5-4d99-9fdd-8182f964c144" containerName="perses"
Oct 30 18:59:00.176573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176338    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="20b9ecc8-f39f-4af5-9bc8-da903375047b" containerName="ta-container"
Oct 30 18:59:00.176573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.176347    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f634f01-e0eb-4285-8624-c3c6d6e54e11" containerName="ta-container"
Oct 30 18:59:00.198834 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.198809    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"]
Oct 30 18:59:00.199003 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.198897    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"
Oct 30 18:59:00.215433 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.215412    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 18:59:00.275273 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.275235    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tctnv\" (UniqueName: \"kubernetes.io/projected/53ee2a0d-f04f-4588-a331-19f7517ee7c4-kube-api-access-tctnv\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf\" (UID: \"53ee2a0d-f04f-4588-a331-19f7517ee7c4\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"
Oct 30 18:59:00.376448 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.376412    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tctnv\" (UniqueName: \"kubernetes.io/projected/53ee2a0d-f04f-4588-a331-19f7517ee7c4-kube-api-access-tctnv\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf\" (UID: \"53ee2a0d-f04f-4588-a331-19f7517ee7c4\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"
Oct 30 18:59:00.402040 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.401978    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tctnv\" (UniqueName: \"kubernetes.io/projected/53ee2a0d-f04f-4588-a331-19f7517ee7c4-kube-api-access-tctnv\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf\" (UID: \"53ee2a0d-f04f-4588-a331-19f7517ee7c4\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"
Oct 30 18:59:00.510957 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.510919    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"
Oct 30 18:59:00.810565 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:00.810539    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"]
Oct 30 18:59:00.812325 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:59:00.812297    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53ee2a0d_f04f_4588_a331_19f7517ee7c4.slice/crio-b74d1c9e490f35d9842065c5d56361266ad94d0ef47e5a87569c5a4a331a0c27 WatchSource:0}: Error finding container b74d1c9e490f35d9842065c5d56361266ad94d0ef47e5a87569c5a4a331a0c27: Status 404 returned error can't find the container with id b74d1c9e490f35d9842065c5d56361266ad94d0ef47e5a87569c5a4a331a0c27
Oct 30 18:59:01.133102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.133065    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf" event={"ID":"53ee2a0d-f04f-4588-a331-19f7517ee7c4","Type":"ContainerStarted","Data":"f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11"}
Oct 30 18:59:01.133102 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.133105    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf" event={"ID":"53ee2a0d-f04f-4588-a331-19f7517ee7c4","Type":"ContainerStarted","Data":"b74d1c9e490f35d9842065c5d56361266ad94d0ef47e5a87569c5a4a331a0c27"}
Oct 30 18:59:01.175705 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.175654    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf" podStartSLOduration=1.175641686 podStartE2EDuration="1.175641686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:01.175198612 +0000 UTC m=+3711.342016763" watchObservedRunningTime="2026-10-30 18:59:01.175641686 +0000 UTC m=+3711.342459829"
Oct 30 18:59:01.187608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.187581    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:59:01.190792 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.190771    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.209896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.209871    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:59:01.269267 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.269237    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 18:59:01.275708 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.275682    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"]
Oct 30 18:59:01.278784 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.278763    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.283823 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.283801    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dctm9\" (UniqueName: \"kubernetes.io/projected/c259f9fd-33e5-4421-baa5-48f424209380-kube-api-access-dctm9\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.283958 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.283901    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/c259f9fd-33e5-4421-baa5-48f424209380-plugins\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.284025 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.283970    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c259f9fd-33e5-4421-baa5-48f424209380-config\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.301064 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.301034    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"]
Oct 30 18:59:01.332855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.332829    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 18:59:01.333150 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.333133    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-qjpth\""
Oct 30 18:59:01.369423 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.369397    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 18:59:01.385409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.385332    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/c259f9fd-33e5-4421-baa5-48f424209380-plugins\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.385409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.385371    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c259f9fd-33e5-4421-baa5-48f424209380-config\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.385409 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.385403    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f8cce3e-2015-4613-88af-fad714663ef0-ta-internal\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.385684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.385424    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f8cce3e-2015-4613-88af-fad714663ef0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.385684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.385607    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dctm9\" (UniqueName: \"kubernetes.io/projected/c259f9fd-33e5-4421-baa5-48f424209380-kube-api-access-dctm9\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.385684 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.385650    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4zfk\" (UniqueName: \"kubernetes.io/projected/3f8cce3e-2015-4613-88af-fad714663ef0-kube-api-access-n4zfk\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.385843 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.385795    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/c259f9fd-33e5-4421-baa5-48f424209380-plugins\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.386100 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.386082    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c259f9fd-33e5-4421-baa5-48f424209380-config\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.410602 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.410578    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dctm9\" (UniqueName: \"kubernetes.io/projected/c259f9fd-33e5-4421-baa5-48f424209380-kube-api-access-dctm9\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:01.486628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.486594    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f8cce3e-2015-4613-88af-fad714663ef0-ta-internal\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.486628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.486630    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f8cce3e-2015-4613-88af-fad714663ef0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.486873 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.486669    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4zfk\" (UniqueName: \"kubernetes.io/projected/3f8cce3e-2015-4613-88af-fad714663ef0-kube-api-access-n4zfk\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.487357 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.487323    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f8cce3e-2015-4613-88af-fad714663ef0-ta-internal\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.489451 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.489428    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f8cce3e-2015-4613-88af-fad714663ef0-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.512055 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.512033    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4zfk\" (UniqueName: \"kubernetes.io/projected/3f8cce3e-2015-4613-88af-fad714663ef0-kube-api-access-n4zfk\") pod \"data-science-collector-targetallocator-775555cbf4-cthxh\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.590105 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.590078    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:01.890691 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.890668    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"]
Oct 30 18:59:01.892314 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:59:01.892261    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f8cce3e_2015_4613_88af_fad714663ef0.slice/crio-020402ea8bbe6d8cd38c4c61b5c95d67c34416340fe46c2943141b20926d2973 WatchSource:0}: Error finding container 020402ea8bbe6d8cd38c4c61b5c95d67c34416340fe46c2943141b20926d2973: Status 404 returned error can't find the container with id 020402ea8bbe6d8cd38c4c61b5c95d67c34416340fe46c2943141b20926d2973
Oct 30 18:59:01.894383 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.894350    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:59:01.894454 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:01.894408    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:59:02.138841 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:02.138804    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh" event={"ID":"3f8cce3e-2015-4613-88af-fad714663ef0","Type":"ContainerStarted","Data":"fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199"}
Oct 30 18:59:02.138841 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:02.138845    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh" event={"ID":"3f8cce3e-2015-4613-88af-fad714663ef0","Type":"ContainerStarted","Data":"020402ea8bbe6d8cd38c4c61b5c95d67c34416340fe46c2943141b20926d2973"}
Oct 30 18:59:02.139090 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:02.138889    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:02.140121 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:02.140095    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh" podUID="3f8cce3e-2015-4613-88af-fad714663ef0" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.174:8080/readyz\": dial tcp 10.133.0.174:8080: connect: connection refused"
Oct 30 18:59:02.183017 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:02.182934    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh" podStartSLOduration=1.182902454 podStartE2EDuration="1.182902454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:02.182387814 +0000 UTC m=+3712.349205955" watchObservedRunningTime="2026-10-30 18:59:02.182902454 +0000 UTC m=+3712.349720580"
Oct 30 18:59:03.143846 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:03.143819    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:07.937374 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:07.937342    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:07.971701 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:07.971676    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:08.038682 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:08.038655    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:08.038855 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:08.038748    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:08.054472 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:08.054443    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:08.159061 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:08.159032    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:08.191341 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:08.191281    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:08.403330 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:08.403292    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:08.709462 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:08.709422    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:59:08.711307 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:59:08.711275    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc259f9fd_33e5_4421_baa5_48f424209380.slice/crio-69d8a564523c44f184c9c8010b6e9d78f7537d15e52823a4ab0d953860e81662 WatchSource:0}: Error finding container 69d8a564523c44f184c9c8010b6e9d78f7537d15e52823a4ab0d953860e81662: Status 404 returned error can't find the container with id 69d8a564523c44f184c9c8010b6e9d78f7537d15e52823a4ab0d953860e81662
Oct 30 18:59:09.169396 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:09.169357    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"c259f9fd-33e5-4421-baa5-48f424209380","Type":"ContainerStarted","Data":"17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c"}
Oct 30 18:59:09.169827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:09.169407    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"c259f9fd-33e5-4421-baa5-48f424209380","Type":"ContainerStarted","Data":"69d8a564523c44f184c9c8010b6e9d78f7537d15e52823a4ab0d953860e81662"}
Oct 30 18:59:09.169827 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:09.169499    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:09.213517 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:09.213473    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=8.213461068 podStartE2EDuration="8.213461068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:59:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:09.212705165 +0000 UTC m=+3719.379523312" watchObservedRunningTime="2026-10-30 18:59:09.213461068 +0000 UTC m=+3719.380279195"
Oct 30 18:59:09.597393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:09.597347    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"]
Oct 30 18:59:09.597607 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:09.597572    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf" podUID="53ee2a0d-f04f-4588-a331-19f7517ee7c4" containerName="thanos-querier" containerID="cri-o://f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11" gracePeriod=30
Oct 30 18:59:09.874876 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:09.874841    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"
Oct 30 18:59:10.058964 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.058932    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53ee2a0d-f04f-4588-a331-19f7517ee7c4-kube-api-access-tctnv\" (UniqueName: \"kubernetes.io/projected/53ee2a0d-f04f-4588-a331-19f7517ee7c4-kube-api-access-tctnv\") pod \"53ee2a0d-f04f-4588-a331-19f7517ee7c4\" (UID: \"53ee2a0d-f04f-4588-a331-19f7517ee7c4\") "
Oct 30 18:59:10.061533 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.061503    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53ee2a0d-f04f-4588-a331-19f7517ee7c4-kube-api-access-tctnv" pod "53ee2a0d-f04f-4588-a331-19f7517ee7c4" (UID: "53ee2a0d-f04f-4588-a331-19f7517ee7c4"). InnerVolumeSpecName "kube-api-access-tctnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:10.140694 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.140614    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:59:10.160393 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.160360    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tctnv\" (UniqueName: \"kubernetes.io/projected/53ee2a0d-f04f-4588-a331-19f7517ee7c4-kube-api-access-tctnv\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:10.176811 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.176777    5746 generic.go:356] "Generic (PLEG): container finished" podID="53ee2a0d-f04f-4588-a331-19f7517ee7c4" containerID="f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11" exitCode=0
Oct 30 18:59:10.177266 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.176813    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"
Oct 30 18:59:10.177266 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.176856    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf" event={"ID":"53ee2a0d-f04f-4588-a331-19f7517ee7c4","Type":"ContainerDied","Data":"f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11"}
Oct 30 18:59:10.177266 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.176902    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf" event={"ID":"53ee2a0d-f04f-4588-a331-19f7517ee7c4","Type":"ContainerDied","Data":"b74d1c9e490f35d9842065c5d56361266ad94d0ef47e5a87569c5a4a331a0c27"}
Oct 30 18:59:10.177266 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.176951    5746 scope.go:122] "RemoveContainer" containerID="f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11"
Oct 30 18:59:10.186050 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.186004    5746 scope.go:122] "RemoveContainer" containerID="f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11"
Oct 30 18:59:10.186363 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:59:10.186337    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11\": container with ID starting with f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11 not found: ID does not exist" containerID="f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11"
Oct 30 18:59:10.186447 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.186373    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11"} err="failed to get container status \"f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11\": rpc error: code = NotFound desc = could not find container \"f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11\": container with ID starting with f3ac885b2639499eb096a57f184342e2ade3555277e943f3b1035d7e2ae1bc11 not found: ID does not exist"
Oct 30 18:59:10.230391 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.230359    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"]
Oct 30 18:59:10.234849 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:10.234817    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-7t7pf"]
Oct 30 18:59:12.007743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:12.007709    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53ee2a0d-f04f-4588-a331-19f7517ee7c4" path="/var/lib/kubelet/pods/53ee2a0d-f04f-4588-a331-19f7517ee7c4/volumes"
Oct 30 18:59:12.789980 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:59:12.789949    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:59:20.140490 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:20.140454    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:59:20.178817 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:20.178790    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:30.140292 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:30.140250    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:59:32.548637 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.548602    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"]
Oct 30 18:59:32.549057 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.548882    5746 cpu_manager.go:414] "containerMap: removing container" podUID="53ee2a0d-f04f-4588-a331-19f7517ee7c4" containerName="thanos-querier"
Oct 30 18:59:32.549057 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.548893    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53ee2a0d-f04f-4588-a331-19f7517ee7c4" containerName="thanos-querier"
Oct 30 18:59:32.549057 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.548964    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="53ee2a0d-f04f-4588-a331-19f7517ee7c4" containerName="thanos-querier"
Oct 30 18:59:32.564441 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.564417    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.575388 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.575359    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"]
Oct 30 18:59:32.654548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.654460    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djnn7\" (UniqueName: \"kubernetes.io/projected/6d39a0fa-3f59-483a-8cd3-9e960ee00003-kube-api-access-djnn7\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.654548 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.654518    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/6d39a0fa-3f59-483a-8cd3-9e960ee00003-ta-internal\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.654774 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.654640    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6d39a0fa-3f59-483a-8cd3-9e960ee00003-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.755756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.755721    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/6d39a0fa-3f59-483a-8cd3-9e960ee00003-ta-internal\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.755970 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.755769    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6d39a0fa-3f59-483a-8cd3-9e960ee00003-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.755970 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.755890    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djnn7\" (UniqueName: \"kubernetes.io/projected/6d39a0fa-3f59-483a-8cd3-9e960ee00003-kube-api-access-djnn7\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.756384 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.756361    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/6d39a0fa-3f59-483a-8cd3-9e960ee00003-ta-internal\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.758443 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.758423    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6d39a0fa-3f59-483a-8cd3-9e960ee00003-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.781058 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.781025    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djnn7\" (UniqueName: \"kubernetes.io/projected/6d39a0fa-3f59-483a-8cd3-9e960ee00003-kube-api-access-djnn7\") pod \"data-science-collector-targetallocator-78f6745d9f-2vhh6\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:32.876195 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:32.876151    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:33.174573 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:33.174523    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"]
Oct 30 18:59:33.176286 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:59:33.176262    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d39a0fa_3f59_483a_8cd3_9e960ee00003.slice/crio-c40be1d76971179f69e5044f9cc3508f9df9d1dc95f892d90f6ce716083db0fb WatchSource:0}: Error finding container c40be1d76971179f69e5044f9cc3508f9df9d1dc95f892d90f6ce716083db0fb: Status 404 returned error can't find the container with id c40be1d76971179f69e5044f9cc3508f9df9d1dc95f892d90f6ce716083db0fb
Oct 30 18:59:33.178669 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:33.178639    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:59:33.178742 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:33.178693    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:59:33.267794 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:33.267763    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6" event={"ID":"6d39a0fa-3f59-483a-8cd3-9e960ee00003","Type":"ContainerStarted","Data":"eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5"}
Oct 30 18:59:33.268012 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:33.267803    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6" event={"ID":"6d39a0fa-3f59-483a-8cd3-9e960ee00003","Type":"ContainerStarted","Data":"c40be1d76971179f69e5044f9cc3508f9df9d1dc95f892d90f6ce716083db0fb"}
Oct 30 18:59:33.268086 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:33.268017    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:33.269518 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:33.269490    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6" podUID="6d39a0fa-3f59-483a-8cd3-9e960ee00003" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.175:8080/readyz\": dial tcp 10.133.0.175:8080: connect: connection refused"
Oct 30 18:59:33.314007 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:33.313955    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6" podStartSLOduration=1.313938294 podStartE2EDuration="1.313938294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:33.312129483 +0000 UTC m=+3743.478947612" watchObservedRunningTime="2026-10-30 18:59:33.313938294 +0000 UTC m=+3743.480756422"
Oct 30 18:59:34.273194 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.273170    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:34.349176 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.349144    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"]
Oct 30 18:59:34.349356 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.349316    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh" podUID="3f8cce3e-2015-4613-88af-fad714663ef0" containerName="ta-container" containerID="cri-o://fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199" gracePeriod=30
Oct 30 18:59:34.618628 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.618600    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:34.774687 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.774655    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f8cce3e-2015-4613-88af-fad714663ef0-kube-api-access-n4zfk\" (UniqueName: \"kubernetes.io/projected/3f8cce3e-2015-4613-88af-fad714663ef0-kube-api-access-n4zfk\") pod \"3f8cce3e-2015-4613-88af-fad714663ef0\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") "
Oct 30 18:59:34.774864 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.774716    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f8cce3e-2015-4613-88af-fad714663ef0-ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f8cce3e-2015-4613-88af-fad714663ef0-ta-internal\") pod \"3f8cce3e-2015-4613-88af-fad714663ef0\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") "
Oct 30 18:59:34.774864 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.774746    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f8cce3e-2015-4613-88af-fad714663ef0-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f8cce3e-2015-4613-88af-fad714663ef0-data-science-collector-ta-server-cert\") pod \"3f8cce3e-2015-4613-88af-fad714663ef0\" (UID: \"3f8cce3e-2015-4613-88af-fad714663ef0\") "
Oct 30 18:59:34.775126 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.775098    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f8cce3e-2015-4613-88af-fad714663ef0-ta-internal" pod "3f8cce3e-2015-4613-88af-fad714663ef0" (UID: "3f8cce3e-2015-4613-88af-fad714663ef0"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:34.776958 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.776902    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f8cce3e-2015-4613-88af-fad714663ef0-data-science-collector-ta-server-cert" pod "3f8cce3e-2015-4613-88af-fad714663ef0" (UID: "3f8cce3e-2015-4613-88af-fad714663ef0"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:34.777037 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.776959    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f8cce3e-2015-4613-88af-fad714663ef0-kube-api-access-n4zfk" pod "3f8cce3e-2015-4613-88af-fad714663ef0" (UID: "3f8cce3e-2015-4613-88af-fad714663ef0"). InnerVolumeSpecName "kube-api-access-n4zfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:34.875569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.875484    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/3f8cce3e-2015-4613-88af-fad714663ef0-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:34.875569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.875517    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/3f8cce3e-2015-4613-88af-fad714663ef0-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:34.875569 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:34.875527    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4zfk\" (UniqueName: \"kubernetes.io/projected/3f8cce3e-2015-4613-88af-fad714663ef0-kube-api-access-n4zfk\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:35.276470 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.276385    5746 generic.go:356] "Generic (PLEG): container finished" podID="3f8cce3e-2015-4613-88af-fad714663ef0" containerID="fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199" exitCode=0
Oct 30 18:59:35.276470 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.276435    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"
Oct 30 18:59:35.276939 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.276478    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh" event={"ID":"3f8cce3e-2015-4613-88af-fad714663ef0","Type":"ContainerDied","Data":"fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199"}
Oct 30 18:59:35.276939 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.276520    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh" event={"ID":"3f8cce3e-2015-4613-88af-fad714663ef0","Type":"ContainerDied","Data":"020402ea8bbe6d8cd38c4c61b5c95d67c34416340fe46c2943141b20926d2973"}
Oct 30 18:59:35.276939 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.276541    5746 scope.go:122] "RemoveContainer" containerID="fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199"
Oct 30 18:59:35.285120 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.285102    5746 scope.go:122] "RemoveContainer" containerID="fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199"
Oct 30 18:59:35.285395 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:59:35.285376    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199\": container with ID starting with fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199 not found: ID does not exist" containerID="fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199"
Oct 30 18:59:35.285452 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.285402    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199"} err="failed to get container status \"fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199\": rpc error: code = NotFound desc = could not find container \"fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199\": container with ID starting with fd1304f3ee57d8f8d27f1c63a05b04b6b526f2436f0e55a4d1e23056d1d21199 not found: ID does not exist"
Oct 30 18:59:35.331610 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.331579    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"]
Oct 30 18:59:35.345510 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:35.345490    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-775555cbf4-cthxh"]
Oct 30 18:59:36.008150 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:36.008111    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f8cce3e-2015-4613-88af-fad714663ef0" path="/var/lib/kubelet/pods/3f8cce3e-2015-4613-88af-fad714663ef0/volumes"
Oct 30 18:59:40.140922 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:40.140866    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:59:42.790022 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:59:42.789982    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 18:59:49.559578 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.559541    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"]
Oct 30 18:59:49.560134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.559851    5746 cpu_manager.go:414] "containerMap: removing container" podUID="3f8cce3e-2015-4613-88af-fad714663ef0" containerName="ta-container"
Oct 30 18:59:49.560134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.559862    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f8cce3e-2015-4613-88af-fad714663ef0" containerName="ta-container"
Oct 30 18:59:49.560134 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.559947    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f8cce3e-2015-4613-88af-fad714663ef0" containerName="ta-container"
Oct 30 18:59:49.565128 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.565108    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.596229 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.596206    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"]
Oct 30 18:59:49.596756 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.596733    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/871c47ba-cad7-4abe-afa4-b618223509ef-ta-internal\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.596812 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.596776    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-479q7\" (UniqueName: \"kubernetes.io/projected/871c47ba-cad7-4abe-afa4-b618223509ef-kube-api-access-479q7\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.596896 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.596875    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/871c47ba-cad7-4abe-afa4-b618223509ef-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.697866 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.697834    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/871c47ba-cad7-4abe-afa4-b618223509ef-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.698031 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.697927    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/871c47ba-cad7-4abe-afa4-b618223509ef-ta-internal\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.698031 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.697951    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-479q7\" (UniqueName: \"kubernetes.io/projected/871c47ba-cad7-4abe-afa4-b618223509ef-kube-api-access-479q7\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.698670 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.698597    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/871c47ba-cad7-4abe-afa4-b618223509ef-ta-internal\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.700501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.700481    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/871c47ba-cad7-4abe-afa4-b618223509ef-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.742626 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.742605    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-479q7\" (UniqueName: \"kubernetes.io/projected/871c47ba-cad7-4abe-afa4-b618223509ef-kube-api-access-479q7\") pod \"data-science-collector-targetallocator-545dc9f6c6-gzdsp\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:49.876697 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:49.876659    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:50.140608 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.140535    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:59:50.176115 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.176087    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"]
Oct 30 18:59:50.178111 ip-10-0-131-31 kubenswrapper[5746]: W1030 18:59:50.178085    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod871c47ba_cad7_4abe_afa4_b618223509ef.slice/crio-46bbfe7d27c9378c5507c26bf616402966cbacca6c44620aafde29c793046cd8 WatchSource:0}: Error finding container 46bbfe7d27c9378c5507c26bf616402966cbacca6c44620aafde29c793046cd8: Status 404 returned error can't find the container with id 46bbfe7d27c9378c5507c26bf616402966cbacca6c44620aafde29c793046cd8
Oct 30 18:59:50.180747 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.180718    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:59:50.180804 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.180773    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 18:59:50.334463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.334429    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp" event={"ID":"871c47ba-cad7-4abe-afa4-b618223509ef","Type":"ContainerStarted","Data":"bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd"}
Oct 30 18:59:50.334463 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.334468    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp" event={"ID":"871c47ba-cad7-4abe-afa4-b618223509ef","Type":"ContainerStarted","Data":"46bbfe7d27c9378c5507c26bf616402966cbacca6c44620aafde29c793046cd8"}
Oct 30 18:59:50.334674 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.334581    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:50.335525 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.335499    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp" podUID="871c47ba-cad7-4abe-afa4-b618223509ef" containerName="ta-container" probeResult="failure" output="Get \"http://10.133.0.176:8080/readyz\": dial tcp 10.133.0.176:8080: connect: connection refused"
Oct 30 18:59:50.378792 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:50.378744    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp" podStartSLOduration=1.378732926 podStartE2EDuration="1.378732926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:50.378215956 +0000 UTC m=+3760.545034084" watchObservedRunningTime="2026-10-30 18:59:50.378732926 +0000 UTC m=+3760.545551052"
Oct 30 18:59:51.340091 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.340064    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:51.414639 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.414603    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"]
Oct 30 18:59:51.414824 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.414766    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6" podUID="6d39a0fa-3f59-483a-8cd3-9e960ee00003" containerName="ta-container" containerID="cri-o://eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5" gracePeriod=30
Oct 30 18:59:51.687575 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.687551    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:51.712948 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.712899    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d39a0fa-3f59-483a-8cd3-9e960ee00003-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6d39a0fa-3f59-483a-8cd3-9e960ee00003-data-science-collector-ta-server-cert\") pod \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") "
Oct 30 18:59:51.713081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.712993    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d39a0fa-3f59-483a-8cd3-9e960ee00003-kube-api-access-djnn7\" (UniqueName: \"kubernetes.io/projected/6d39a0fa-3f59-483a-8cd3-9e960ee00003-kube-api-access-djnn7\") pod \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") "
Oct 30 18:59:51.713081 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.713029    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d39a0fa-3f59-483a-8cd3-9e960ee00003-ta-internal\" (UniqueName: \"kubernetes.io/configmap/6d39a0fa-3f59-483a-8cd3-9e960ee00003-ta-internal\") pod \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\" (UID: \"6d39a0fa-3f59-483a-8cd3-9e960ee00003\") "
Oct 30 18:59:51.713396 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.713371    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d39a0fa-3f59-483a-8cd3-9e960ee00003-ta-internal" pod "6d39a0fa-3f59-483a-8cd3-9e960ee00003" (UID: "6d39a0fa-3f59-483a-8cd3-9e960ee00003"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:51.715042 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.715019    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d39a0fa-3f59-483a-8cd3-9e960ee00003-data-science-collector-ta-server-cert" pod "6d39a0fa-3f59-483a-8cd3-9e960ee00003" (UID: "6d39a0fa-3f59-483a-8cd3-9e960ee00003"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:51.715174 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.715157    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d39a0fa-3f59-483a-8cd3-9e960ee00003-kube-api-access-djnn7" pod "6d39a0fa-3f59-483a-8cd3-9e960ee00003" (UID: "6d39a0fa-3f59-483a-8cd3-9e960ee00003"). InnerVolumeSpecName "kube-api-access-djnn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:51.813730 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.813697    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/6d39a0fa-3f59-483a-8cd3-9e960ee00003-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:51.813730 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.813723    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djnn7\" (UniqueName: \"kubernetes.io/projected/6d39a0fa-3f59-483a-8cd3-9e960ee00003-kube-api-access-djnn7\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:51.813730 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:51.813733    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/6d39a0fa-3f59-483a-8cd3-9e960ee00003-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:52.343523 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.343435    5746 generic.go:356] "Generic (PLEG): container finished" podID="6d39a0fa-3f59-483a-8cd3-9e960ee00003" containerID="eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5" exitCode=0
Oct 30 18:59:52.343523 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.343498    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"
Oct 30 18:59:52.344024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.343524    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6" event={"ID":"6d39a0fa-3f59-483a-8cd3-9e960ee00003","Type":"ContainerDied","Data":"eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5"}
Oct 30 18:59:52.344024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.343569    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6" event={"ID":"6d39a0fa-3f59-483a-8cd3-9e960ee00003","Type":"ContainerDied","Data":"c40be1d76971179f69e5044f9cc3508f9df9d1dc95f892d90f6ce716083db0fb"}
Oct 30 18:59:52.344024 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.343585    5746 scope.go:122] "RemoveContainer" containerID="eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5"
Oct 30 18:59:52.351692 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.351668    5746 scope.go:122] "RemoveContainer" containerID="eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5"
Oct 30 18:59:52.351994 ip-10-0-131-31 kubenswrapper[5746]: E1030 18:59:52.351974    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5\": container with ID starting with eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5 not found: ID does not exist" containerID="eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5"
Oct 30 18:59:52.352045 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.352013    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5"} err="failed to get container status \"eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5\": rpc error: code = NotFound desc = could not find container \"eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5\": container with ID starting with eb773e71f0adef7eaaf83fc5de68bb15c051f4516d11198ca13e1d5f4efb74e5 not found: ID does not exist"
Oct 30 18:59:52.399301 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.399277    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"]
Oct 30 18:59:52.412777 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:52.412752    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-78f6745d9f-2vhh6"]
Oct 30 18:59:54.008501 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:54.008470    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d39a0fa-3f59-483a-8cd3-9e960ee00003" path="/var/lib/kubelet/pods/6d39a0fa-3f59-483a-8cd3-9e960ee00003/volumes"
Oct 30 18:59:59.380773 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.380740    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"]
Oct 30 18:59:59.381294 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.380971    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp" podUID="871c47ba-cad7-4abe-afa4-b618223509ef" containerName="ta-container" containerID="cri-o://bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd" gracePeriod=30
Oct 30 18:59:59.488182 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.488148    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 18:59:59.488533 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.488366    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="c259f9fd-33e5-4421-baa5-48f424209380" containerName="perses" containerID="cri-o://17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c" gracePeriod=30
Oct 30 18:59:59.712159 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.712132    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 18:59:59.778743 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.778708    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/871c47ba-cad7-4abe-afa4-b618223509ef-ta-internal\" (UniqueName: \"kubernetes.io/configmap/871c47ba-cad7-4abe-afa4-b618223509ef-ta-internal\") pod \"871c47ba-cad7-4abe-afa4-b618223509ef\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") "
Oct 30 18:59:59.779041 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.778762    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/871c47ba-cad7-4abe-afa4-b618223509ef-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/871c47ba-cad7-4abe-afa4-b618223509ef-data-science-collector-ta-server-cert\") pod \"871c47ba-cad7-4abe-afa4-b618223509ef\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") "
Oct 30 18:59:59.779041 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.778813    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/871c47ba-cad7-4abe-afa4-b618223509ef-kube-api-access-479q7\" (UniqueName: \"kubernetes.io/projected/871c47ba-cad7-4abe-afa4-b618223509ef-kube-api-access-479q7\") pod \"871c47ba-cad7-4abe-afa4-b618223509ef\" (UID: \"871c47ba-cad7-4abe-afa4-b618223509ef\") "
Oct 30 18:59:59.779459 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.779423    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/871c47ba-cad7-4abe-afa4-b618223509ef-ta-internal" pod "871c47ba-cad7-4abe-afa4-b618223509ef" (UID: "871c47ba-cad7-4abe-afa4-b618223509ef"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:59.781629 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.781589    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/871c47ba-cad7-4abe-afa4-b618223509ef-data-science-collector-ta-server-cert" pod "871c47ba-cad7-4abe-afa4-b618223509ef" (UID: "871c47ba-cad7-4abe-afa4-b618223509ef"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:59.781768 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.781740    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/871c47ba-cad7-4abe-afa4-b618223509ef-kube-api-access-479q7" pod "871c47ba-cad7-4abe-afa4-b618223509ef" (UID: "871c47ba-cad7-4abe-afa4-b618223509ef"). InnerVolumeSpecName "kube-api-access-479q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:59.787218 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.787193    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/871c47ba-cad7-4abe-afa4-b618223509ef-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:59.787312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.787225    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/871c47ba-cad7-4abe-afa4-b618223509ef-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:59.787312 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.787243    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-479q7\" (UniqueName: \"kubernetes.io/projected/871c47ba-cad7-4abe-afa4-b618223509ef-kube-api-access-479q7\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:59.797545 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.797520    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 18:59:59.888188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.888159    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c259f9fd-33e5-4421-baa5-48f424209380-kube-api-access-dctm9\" (UniqueName: \"kubernetes.io/projected/c259f9fd-33e5-4421-baa5-48f424209380-kube-api-access-dctm9\") pod \"c259f9fd-33e5-4421-baa5-48f424209380\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") "
Oct 30 18:59:59.888188 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.888194    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c259f9fd-33e5-4421-baa5-48f424209380-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c259f9fd-33e5-4421-baa5-48f424209380-plugins\") pod \"c259f9fd-33e5-4421-baa5-48f424209380\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") "
Oct 30 18:59:59.888450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.888223    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c259f9fd-33e5-4421-baa5-48f424209380-config\" (UniqueName: \"kubernetes.io/configmap/c259f9fd-33e5-4421-baa5-48f424209380-config\") pod \"c259f9fd-33e5-4421-baa5-48f424209380\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") "
Oct 30 18:59:59.888450 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.888363    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"c259f9fd-33e5-4421-baa5-48f424209380\" (UID: \"c259f9fd-33e5-4421-baa5-48f424209380\") "
Oct 30 18:59:59.888669 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.888643    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c259f9fd-33e5-4421-baa5-48f424209380-config" pod "c259f9fd-33e5-4421-baa5-48f424209380" (UID: "c259f9fd-33e5-4421-baa5-48f424209380"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:59.890435 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.890413    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c259f9fd-33e5-4421-baa5-48f424209380-kube-api-access-dctm9" pod "c259f9fd-33e5-4421-baa5-48f424209380" (UID: "c259f9fd-33e5-4421-baa5-48f424209380"). InnerVolumeSpecName "kube-api-access-dctm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:59.891170 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.891153    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "c259f9fd-33e5-4421-baa5-48f424209380" (UID: "c259f9fd-33e5-4421-baa5-48f424209380"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 18:59:59.989758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.989680    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dctm9\" (UniqueName: \"kubernetes.io/projected/c259f9fd-33e5-4421-baa5-48f424209380-kube-api-access-dctm9\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:59.989758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.989708    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c259f9fd-33e5-4421-baa5-48f424209380-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 18:59:59.989758 ip-10-0-131-31 kubenswrapper[5746]: I1030 18:59:59.989743    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 19:00:00.002964 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.002944    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 19:00:00.038663 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.038630    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c259f9fd-33e5-4421-baa5-48f424209380-plugins" pod "c259f9fd-33e5-4421-baa5-48f424209380" (UID: "c259f9fd-33e5-4421-baa5-48f424209380"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:00.091095 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.091060    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/c259f9fd-33e5-4421-baa5-48f424209380-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 19:00:00.091095 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.091087    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 19:00:00.140148 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.140103    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:00.378492 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.378460    5746 generic.go:356] "Generic (PLEG): container finished" podID="871c47ba-cad7-4abe-afa4-b618223509ef" containerID="bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd" exitCode=0
Oct 30 19:00:00.378705 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.378518    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"
Oct 30 19:00:00.378705 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.378543    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp" event={"ID":"871c47ba-cad7-4abe-afa4-b618223509ef","Type":"ContainerDied","Data":"bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd"}
Oct 30 19:00:00.378705 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.378586    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp" event={"ID":"871c47ba-cad7-4abe-afa4-b618223509ef","Type":"ContainerDied","Data":"46bbfe7d27c9378c5507c26bf616402966cbacca6c44620aafde29c793046cd8"}
Oct 30 19:00:00.378705 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.378612    5746 scope.go:122] "RemoveContainer" containerID="bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd"
Oct 30 19:00:00.383582 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.380342    5746 generic.go:356] "Generic (PLEG): container finished" podID="c259f9fd-33e5-4421-baa5-48f424209380" containerID="17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c" exitCode=0
Oct 30 19:00:00.383582 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.380510    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"c259f9fd-33e5-4421-baa5-48f424209380","Type":"ContainerDied","Data":"17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c"}
Oct 30 19:00:00.383582 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.380541    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"c259f9fd-33e5-4421-baa5-48f424209380","Type":"ContainerDied","Data":"69d8a564523c44f184c9c8010b6e9d78f7537d15e52823a4ab0d953860e81662"}
Oct 30 19:00:00.383582 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.380605    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:00.390474 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.390299    5746 scope.go:122] "RemoveContainer" containerID="bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd"
Oct 30 19:00:00.390699 ip-10-0-131-31 kubenswrapper[5746]: E1030 19:00:00.390671    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd\": container with ID starting with bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd not found: ID does not exist" containerID="bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd"
Oct 30 19:00:00.390875 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.390707    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd"} err="failed to get container status \"bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd\": rpc error: code = NotFound desc = could not find container \"bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd\": container with ID starting with bdf5801f03be802264b4ec5ae525996cd3982c0bca54b3c7c5443b6e7747c2fd not found: ID does not exist"
Oct 30 19:00:00.390875 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.390731    5746 scope.go:122] "RemoveContainer" containerID="17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c"
Oct 30 19:00:00.414098 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.414075    5746 scope.go:122] "RemoveContainer" containerID="17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c"
Oct 30 19:00:00.414407 ip-10-0-131-31 kubenswrapper[5746]: E1030 19:00:00.414389    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c\": container with ID starting with 17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c not found: ID does not exist" containerID="17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c"
Oct 30 19:00:00.414467 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.414418    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c"} err="failed to get container status \"17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c\": rpc error: code = NotFound desc = could not find container \"17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c\": container with ID starting with 17b04fc505a5d8aada2181afd99949fa456d0a7983df7195d68e0e96a8ccac1c not found: ID does not exist"
Oct 30 19:00:00.433394 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.433368    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 19:00:00.437696 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.437676    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 19:00:00.475455 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.475421    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"]
Oct 30 19:00:00.480299 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:00.480272    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-545dc9f6c6-gzdsp"]
Oct 30 19:00:02.007885 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:02.007847    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="871c47ba-cad7-4abe-afa4-b618223509ef" path="/var/lib/kubelet/pods/871c47ba-cad7-4abe-afa4-b618223509ef/volumes"
Oct 30 19:00:02.008462 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:02.008441    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c259f9fd-33e5-4421-baa5-48f424209380" path="/var/lib/kubelet/pods/c259f9fd-33e5-4421-baa5-48f424209380/volumes"
Oct 30 19:00:10.140950 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:10.140864    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:12.192204 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192172    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"]
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192459    5746 cpu_manager.go:414] "containerMap: removing container" podUID="6d39a0fa-3f59-483a-8cd3-9e960ee00003" containerName="ta-container"
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192471    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d39a0fa-3f59-483a-8cd3-9e960ee00003" containerName="ta-container"
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192489    5746 cpu_manager.go:414] "containerMap: removing container" podUID="c259f9fd-33e5-4421-baa5-48f424209380" containerName="perses"
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192494    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c259f9fd-33e5-4421-baa5-48f424209380" containerName="perses"
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192504    5746 cpu_manager.go:414] "containerMap: removing container" podUID="871c47ba-cad7-4abe-afa4-b618223509ef" containerName="ta-container"
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192510    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="871c47ba-cad7-4abe-afa4-b618223509ef" containerName="ta-container"
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192559    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d39a0fa-3f59-483a-8cd3-9e960ee00003" containerName="ta-container"
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192568    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="c259f9fd-33e5-4421-baa5-48f424209380" containerName="perses"
Oct 30 19:00:12.192596 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.192576    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="871c47ba-cad7-4abe-afa4-b618223509ef" containerName="ta-container"
Oct 30 19:00:12.197474 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.197454    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"
Oct 30 19:00:12.213077 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.213051    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"]
Oct 30 19:00:12.224061 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.224039    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"default-dockercfg-5mtd6\""
Oct 30 19:00:12.292996 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.292958    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrk4v\" (UniqueName: \"kubernetes.io/projected/ce3472e4-6b27-465f-bd8a-821eb643251c-kube-api-access-jrk4v\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7\" (UID: \"ce3472e4-6b27-465f-bd8a-821eb643251c\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"
Oct 30 19:00:12.393624 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.393588    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrk4v\" (UniqueName: \"kubernetes.io/projected/ce3472e4-6b27-465f-bd8a-821eb643251c-kube-api-access-jrk4v\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7\" (UID: \"ce3472e4-6b27-465f-bd8a-821eb643251c\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"
Oct 30 19:00:12.418065 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.418029    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrk4v\" (UniqueName: \"kubernetes.io/projected/ce3472e4-6b27-465f-bd8a-821eb643251c-kube-api-access-jrk4v\") pod \"thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7\" (UID: \"ce3472e4-6b27-465f-bd8a-821eb643251c\") " pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"
Oct 30 19:00:12.509385 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.509301    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"
Oct 30 19:00:12.791148 ip-10-0-131-31 kubenswrapper[5746]: E1030 19:00:12.791071    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 19:00:12.801775 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:12.801742    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"]
Oct 30 19:00:12.805854 ip-10-0-131-31 kubenswrapper[5746]: W1030 19:00:12.805816    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce3472e4_6b27_465f_bd8a_821eb643251c.slice/crio-ac4dbfe164d7a4a04bf1e2f9dffdd002e38178839e336a5a5b00b3901f014320 WatchSource:0}: Error finding container ac4dbfe164d7a4a04bf1e2f9dffdd002e38178839e336a5a5b00b3901f014320: Status 404 returned error can't find the container with id ac4dbfe164d7a4a04bf1e2f9dffdd002e38178839e336a5a5b00b3901f014320
Oct 30 19:00:13.203579 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.203548    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 19:00:13.206759 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.206738    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.221081 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.221058    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-perses-config\""
Oct 30 19:00:13.227148 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.227119    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 19:00:13.295597 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.295566    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"]
Oct 30 19:00:13.298899 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.298879    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.301348 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.301323    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88ce202b-f466-4add-b735-dd61583d4f6f-config\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.301471 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.301370    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/88ce202b-f466-4add-b735-dd61583d4f6f-plugins\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.301537 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.301468    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w5nt\" (UniqueName: \"kubernetes.io/projected/88ce202b-f466-4add-b735-dd61583d4f6f-kube-api-access-2w5nt\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.315094 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.315069    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator\""
Oct 30 19:00:13.315213 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.315122    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-ta-server-cert\""
Oct 30 19:00:13.315213 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.315138    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ods-monitoring\"/\"data-science-collector-targetallocator-dockercfg-zt9zz\""
Oct 30 19:00:13.320962 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.320889    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"]
Oct 30 19:00:13.402365 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.402333    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w5nt\" (UniqueName: \"kubernetes.io/projected/88ce202b-f466-4add-b735-dd61583d4f6f-kube-api-access-2w5nt\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.402546 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.402375    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd8dj\" (UniqueName: \"kubernetes.io/projected/73f85af5-d985-4e04-8482-f93a59d14d79-kube-api-access-cd8dj\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.402546 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.402497    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/73f85af5-d985-4e04-8482-f93a59d14d79-ta-internal\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.402546 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.402538    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/73f85af5-d985-4e04-8482-f93a59d14d79-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.402677 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.402632    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88ce202b-f466-4add-b735-dd61583d4f6f-config\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.402677 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.402668    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/88ce202b-f466-4add-b735-dd61583d4f6f-plugins\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.403094 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.403079    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/88ce202b-f466-4add-b735-dd61583d4f6f-plugins\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.403335 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.403318    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88ce202b-f466-4add-b735-dd61583d4f6f-config\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.426145 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.426121    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w5nt\" (UniqueName: \"kubernetes.io/projected/88ce202b-f466-4add-b735-dd61583d4f6f-kube-api-access-2w5nt\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:13.434360 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.434329    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7" event={"ID":"ce3472e4-6b27-465f-bd8a-821eb643251c","Type":"ContainerStarted","Data":"e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd"}
Oct 30 19:00:13.434497 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.434375    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7" event={"ID":"ce3472e4-6b27-465f-bd8a-821eb643251c","Type":"ContainerStarted","Data":"ac4dbfe164d7a4a04bf1e2f9dffdd002e38178839e336a5a5b00b3901f014320"}
Oct 30 19:00:13.503543 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.503441    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd8dj\" (UniqueName: \"kubernetes.io/projected/73f85af5-d985-4e04-8482-f93a59d14d79-kube-api-access-cd8dj\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.503543 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.503489    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/73f85af5-d985-4e04-8482-f93a59d14d79-ta-internal\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.503543 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.503519    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/73f85af5-d985-4e04-8482-f93a59d14d79-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.504330 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.504299    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/73f85af5-d985-4e04-8482-f93a59d14d79-ta-internal\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.506318 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.506289    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/73f85af5-d985-4e04-8482-f93a59d14d79-data-science-collector-ta-server-cert\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.529022 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.528991    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd8dj\" (UniqueName: \"kubernetes.io/projected/73f85af5-d985-4e04-8482-f93a59d14d79-kube-api-access-cd8dj\") pod \"data-science-collector-targetallocator-5544bffbc5-ldh4c\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") " pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.610794 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.610762    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:13.906756 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.906702    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7" podStartSLOduration=1.906689148 podStartE2EDuration="1.906689148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:13.475198198 +0000 UTC m=+3783.642016317" watchObservedRunningTime="2026-10-30 19:00:13.906689148 +0000 UTC m=+3784.073507268"
Oct 30 19:00:13.906977 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.906934    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"]
Oct 30 19:00:13.910003 ip-10-0-131-31 kubenswrapper[5746]: W1030 19:00:13.909978    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73f85af5_d985_4e04_8482_f93a59d14d79.slice/crio-5e6243d295002efe9b8e4bbc039a059b81974e05768bbf92a11f69b1aa0f2e89 WatchSource:0}: Error finding container 5e6243d295002efe9b8e4bbc039a059b81974e05768bbf92a11f69b1aa0f2e89: Status 404 returned error can't find the container with id 5e6243d295002efe9b8e4bbc039a059b81974e05768bbf92a11f69b1aa0f2e89
Oct 30 19:00:13.912113 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.912084    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:00:13.912242 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:13.912143    5746 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:00:14.440420 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:14.440384    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c" event={"ID":"73f85af5-d985-4e04-8482-f93a59d14d79","Type":"ContainerStarted","Data":"f831a4f215b1bbcc62116b2d61739cb9354d9fd6870e6b63dbcdd7e6e02f9207"}
Oct 30 19:00:14.440826 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:14.440432    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:14.440826 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:14.440451    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c" event={"ID":"73f85af5-d985-4e04-8482-f93a59d14d79","Type":"ContainerStarted","Data":"5e6243d295002efe9b8e4bbc039a059b81974e05768bbf92a11f69b1aa0f2e89"}
Oct 30 19:00:14.442354 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:14.442333    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:14.505772 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:14.505722    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c" podStartSLOduration=1.505706453 podStartE2EDuration="1.505706453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:14.504809879 +0000 UTC m=+3784.671627998" watchObservedRunningTime="2026-10-30 19:00:14.505706453 +0000 UTC m=+3784.672524580"
Oct 30 19:00:19.454643 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:19.454599    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:19.484050 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:19.484018    5746 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") device path: \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:19.555625 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:19.555586    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:19.555838 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:19.555749    5746 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") DevicePath \"\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:19.570014 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:19.569985    5746 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:19.672528 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:19.672498    5746 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/b077231f830c06ae1d60cbef163599c81efd29f092a0b854551f5619804c537e/globalmount\"" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:19.701381 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:19.701350    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"data-science-perses-0\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") " pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:19.818157 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:19.818070    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:20.111984 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:20.111953    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 19:00:20.115250 ip-10-0-131-31 kubenswrapper[5746]: W1030 19:00:20.115226    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88ce202b_f466_4add_b735_dd61583d4f6f.slice/crio-3235155d2fc723b1f856be6a1af195e6302c57d0d017ef38bbcc83a48ceaf5d7 WatchSource:0}: Error finding container 3235155d2fc723b1f856be6a1af195e6302c57d0d017ef38bbcc83a48ceaf5d7: Status 404 returned error can't find the container with id 3235155d2fc723b1f856be6a1af195e6302c57d0d017ef38bbcc83a48ceaf5d7
Oct 30 19:00:20.140489 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:20.140291    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:20.466231 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:20.466147    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"88ce202b-f466-4add-b735-dd61583d4f6f","Type":"ContainerStarted","Data":"80a3edecd2bd2d64df1572deceb737f28abce0bcf40f9030678e2cb325506ccc"}
Oct 30 19:00:20.466231 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:20.466194    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"88ce202b-f466-4add-b735-dd61583d4f6f","Type":"ContainerStarted","Data":"3235155d2fc723b1f856be6a1af195e6302c57d0d017ef38bbcc83a48ceaf5d7"}
Oct 30 19:00:20.466716 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:20.466271    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:20.507195 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:20.507149    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="redhat-ods-monitoring/data-science-perses-0" podStartSLOduration=7.507136078 podStartE2EDuration="7.507136078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:20.507035881 +0000 UTC m=+3790.673854008" watchObservedRunningTime="2026-10-30 19:00:20.507136078 +0000 UTC m=+3790.673954204"
Oct 30 19:00:21.023684 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.023574    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"]
Oct 30 19:00:21.023984 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.023782    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7" podUID="ce3472e4-6b27-465f-bd8a-821eb643251c" containerName="thanos-querier" containerID="cri-o://e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd" gracePeriod=30
Oct 30 19:00:21.310313 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.310289    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"
Oct 30 19:00:21.369759 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.369730    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce3472e4-6b27-465f-bd8a-821eb643251c-kube-api-access-jrk4v\" (UniqueName: \"kubernetes.io/projected/ce3472e4-6b27-465f-bd8a-821eb643251c-kube-api-access-jrk4v\") pod \"ce3472e4-6b27-465f-bd8a-821eb643251c\" (UID: \"ce3472e4-6b27-465f-bd8a-821eb643251c\") "
Oct 30 19:00:21.372023 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.371989    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce3472e4-6b27-465f-bd8a-821eb643251c-kube-api-access-jrk4v" pod "ce3472e4-6b27-465f-bd8a-821eb643251c" (UID: "ce3472e4-6b27-465f-bd8a-821eb643251c"). InnerVolumeSpecName "kube-api-access-jrk4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:21.470950 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.470892    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrk4v\" (UniqueName: \"kubernetes.io/projected/ce3472e4-6b27-465f-bd8a-821eb643251c-kube-api-access-jrk4v\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 19:00:21.472063 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.472031    5746 generic.go:356] "Generic (PLEG): container finished" podID="ce3472e4-6b27-465f-bd8a-821eb643251c" containerID="e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd" exitCode=0
Oct 30 19:00:21.472212 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.472080    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"
Oct 30 19:00:21.472212 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.472080    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7" event={"ID":"ce3472e4-6b27-465f-bd8a-821eb643251c","Type":"ContainerDied","Data":"e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd"}
Oct 30 19:00:21.472212 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.472121    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7" event={"ID":"ce3472e4-6b27-465f-bd8a-821eb643251c","Type":"ContainerDied","Data":"ac4dbfe164d7a4a04bf1e2f9dffdd002e38178839e336a5a5b00b3901f014320"}
Oct 30 19:00:21.472212 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.472140    5746 scope.go:122] "RemoveContainer" containerID="e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd"
Oct 30 19:00:21.481143 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.481128    5746 scope.go:122] "RemoveContainer" containerID="e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd"
Oct 30 19:00:21.481427 ip-10-0-131-31 kubenswrapper[5746]: E1030 19:00:21.481411    5746 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd\": container with ID starting with e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd not found: ID does not exist" containerID="e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd"
Oct 30 19:00:21.481518 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.481433    5746 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd"} err="failed to get container status \"e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd\": rpc error: code = NotFound desc = could not find container \"e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd\": container with ID starting with e022105aa58caff349c67a74742917618732e6130f0e032e3e49ece250c567cd not found: ID does not exist"
Oct 30 19:00:21.522196 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.522165    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"]
Oct 30 19:00:21.524796 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:21.524770    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/thanos-querier-data-science-thanos-querier-69cb6bdcdc-ppls7"]
Oct 30 19:00:22.016514 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:22.016482    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce3472e4-6b27-465f-bd8a-821eb643251c" path="/var/lib/kubelet/pods/ce3472e4-6b27-465f-bd8a-821eb643251c/volumes"
Oct 30 19:00:30.140576 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:30.140534    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:31.475599 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:31.475568    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:40.140236 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:40.140188    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:42.790389 ip-10-0-131-31 kubenswrapper[5746]: E1030 19:00:42.790353    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 19:00:50.140894 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:50.140852    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:00:51.338184 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.338146    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"]
Oct 30 19:00:51.338758 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.338375    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c" podUID="73f85af5-d985-4e04-8482-f93a59d14d79" containerName="ta-container" containerID="cri-o://f831a4f215b1bbcc62116b2d61739cb9354d9fd6870e6b63dbcdd7e6e02f9207" gracePeriod=30
Oct 30 19:00:51.434637 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.434603    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 19:00:51.434874 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.434824    5746 kuberuntime_container.go:898] "Killing container with a grace period" pod="redhat-ods-monitoring/data-science-perses-0" podUID="88ce202b-f466-4add-b735-dd61583d4f6f" containerName="perses" containerID="cri-o://80a3edecd2bd2d64df1572deceb737f28abce0bcf40f9030678e2cb325506ccc" gracePeriod=30
Oct 30 19:00:51.473320 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.473285    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-perses-0" podUID="88ce202b-f466-4add-b735-dd61583d4f6f" containerName="perses" probeResult="failure" output="Get \"http://10.133.0.178:8080/metrics\": dial tcp 10.133.0.178:8080: connect: connection refused"
Oct 30 19:00:51.590270 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.590167    5746 generic.go:356] "Generic (PLEG): container finished" podID="73f85af5-d985-4e04-8482-f93a59d14d79" containerID="f831a4f215b1bbcc62116b2d61739cb9354d9fd6870e6b63dbcdd7e6e02f9207" exitCode=0
Oct 30 19:00:51.590270 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.590236    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c" event={"ID":"73f85af5-d985-4e04-8482-f93a59d14d79","Type":"ContainerDied","Data":"f831a4f215b1bbcc62116b2d61739cb9354d9fd6870e6b63dbcdd7e6e02f9207"}
Oct 30 19:00:51.591930 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.591881    5746 generic.go:356] "Generic (PLEG): container finished" podID="88ce202b-f466-4add-b735-dd61583d4f6f" containerID="80a3edecd2bd2d64df1572deceb737f28abce0bcf40f9030678e2cb325506ccc" exitCode=0
Oct 30 19:00:51.592083 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.591931    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"88ce202b-f466-4add-b735-dd61583d4f6f","Type":"ContainerDied","Data":"80a3edecd2bd2d64df1572deceb737f28abce0bcf40f9030678e2cb325506ccc"}
Oct 30 19:00:51.644878 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.644848    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:51.732851 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.732820    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73f85af5-d985-4e04-8482-f93a59d14d79-ta-internal\" (UniqueName: \"kubernetes.io/configmap/73f85af5-d985-4e04-8482-f93a59d14d79-ta-internal\") pod \"73f85af5-d985-4e04-8482-f93a59d14d79\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") "
Oct 30 19:00:51.733003 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.732867    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73f85af5-d985-4e04-8482-f93a59d14d79-kube-api-access-cd8dj\" (UniqueName: \"kubernetes.io/projected/73f85af5-d985-4e04-8482-f93a59d14d79-kube-api-access-cd8dj\") pod \"73f85af5-d985-4e04-8482-f93a59d14d79\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") "
Oct 30 19:00:51.733003 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.732958    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73f85af5-d985-4e04-8482-f93a59d14d79-data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/73f85af5-d985-4e04-8482-f93a59d14d79-data-science-collector-ta-server-cert\") pod \"73f85af5-d985-4e04-8482-f93a59d14d79\" (UID: \"73f85af5-d985-4e04-8482-f93a59d14d79\") "
Oct 30 19:00:51.733264 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.733239    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73f85af5-d985-4e04-8482-f93a59d14d79-ta-internal" pod "73f85af5-d985-4e04-8482-f93a59d14d79" (UID: "73f85af5-d985-4e04-8482-f93a59d14d79"). InnerVolumeSpecName "ta-internal". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:51.735145 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.735123    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73f85af5-d985-4e04-8482-f93a59d14d79-kube-api-access-cd8dj" pod "73f85af5-d985-4e04-8482-f93a59d14d79" (UID: "73f85af5-d985-4e04-8482-f93a59d14d79"). InnerVolumeSpecName "kube-api-access-cd8dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:51.735230 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.735153    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73f85af5-d985-4e04-8482-f93a59d14d79-data-science-collector-ta-server-cert" pod "73f85af5-d985-4e04-8482-f93a59d14d79" (UID: "73f85af5-d985-4e04-8482-f93a59d14d79"). InnerVolumeSpecName "data-science-collector-ta-server-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:51.735445 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.735429    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:51.833754 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.833717    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88ce202b-f466-4add-b735-dd61583d4f6f-kube-api-access-2w5nt\" (UniqueName: \"kubernetes.io/projected/88ce202b-f466-4add-b735-dd61583d4f6f-kube-api-access-2w5nt\") pod \"88ce202b-f466-4add-b735-dd61583d4f6f\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") "
Oct 30 19:00:51.833754 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.833760    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88ce202b-f466-4add-b735-dd61583d4f6f-plugins\" (UniqueName: \"kubernetes.io/empty-dir/88ce202b-f466-4add-b735-dd61583d4f6f-plugins\") pod \"88ce202b-f466-4add-b735-dd61583d4f6f\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") "
Oct 30 19:00:51.834068 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.833798    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88ce202b-f466-4add-b735-dd61583d4f6f-config\" (UniqueName: \"kubernetes.io/configmap/88ce202b-f466-4add-b735-dd61583d4f6f-config\") pod \"88ce202b-f466-4add-b735-dd61583d4f6f\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") "
Oct 30 19:00:51.834068 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.833978    5746 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") pod \"88ce202b-f466-4add-b735-dd61583d4f6f\" (UID: \"88ce202b-f466-4add-b735-dd61583d4f6f\") "
Oct 30 19:00:51.834241 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.834221    5746 reconciler_common.go:299] "Volume detached for volume \"data-science-collector-ta-server-cert\" (UniqueName: \"kubernetes.io/secret/73f85af5-d985-4e04-8482-f93a59d14d79-data-science-collector-ta-server-cert\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 19:00:51.834241 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.834231    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88ce202b-f466-4add-b735-dd61583d4f6f-config" pod "88ce202b-f466-4add-b735-dd61583d4f6f" (UID: "88ce202b-f466-4add-b735-dd61583d4f6f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:51.834375 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.834247    5746 reconciler_common.go:299] "Volume detached for volume \"ta-internal\" (UniqueName: \"kubernetes.io/configmap/73f85af5-d985-4e04-8482-f93a59d14d79-ta-internal\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 19:00:51.834375 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.834261    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd8dj\" (UniqueName: \"kubernetes.io/projected/73f85af5-d985-4e04-8482-f93a59d14d79-kube-api-access-cd8dj\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 19:00:51.836225 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.836199    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88ce202b-f466-4add-b735-dd61583d4f6f-kube-api-access-2w5nt" pod "88ce202b-f466-4add-b735-dd61583d4f6f" (UID: "88ce202b-f466-4add-b735-dd61583d4f6f"). InnerVolumeSpecName "kube-api-access-2w5nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:51.836928 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.836881    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0" pod "88ce202b-f466-4add-b735-dd61583d4f6f" (UID: "88ce202b-f466-4add-b735-dd61583d4f6f"). InnerVolumeSpecName "pvc-eed13050-88da-4f09-b402-425b815d318b". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 19:00:51.934621 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.934588    5746 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88ce202b-f466-4add-b735-dd61583d4f6f-config\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 19:00:51.934789 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.934643    5746 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" "
Oct 30 19:00:51.934789 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.934655    5746 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w5nt\" (UniqueName: \"kubernetes.io/projected/88ce202b-f466-4add-b735-dd61583d4f6f-kube-api-access-2w5nt\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 19:00:51.947716 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.947698    5746 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-eed13050-88da-4f09-b402-425b815d318b" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0") on node "ip-10-0-131-31.ec2.internal"
Oct 30 19:00:51.980610 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:51.980581    5746 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88ce202b-f466-4add-b735-dd61583d4f6f-plugins" pod "88ce202b-f466-4add-b735-dd61583d4f6f" (UID: "88ce202b-f466-4add-b735-dd61583d4f6f"). InnerVolumeSpecName "plugins". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:52.035749 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.035723    5746 reconciler_common.go:299] "Volume detached for volume \"pvc-eed13050-88da-4f09-b402-425b815d318b\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-00e8e22ac81decba0\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"csi-846148f0a530cb656b3e03b44288615439759f717fe081e5d2df70c5f7aaef8a\""
Oct 30 19:00:52.035749 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.035748    5746 reconciler_common.go:299] "Volume detached for volume \"plugins\" (UniqueName: \"kubernetes.io/empty-dir/88ce202b-f466-4add-b735-dd61583d4f6f-plugins\") on node \"ip-10-0-131-31.ec2.internal\" DevicePath \"\""
Oct 30 19:00:52.598135 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.598102    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"
Oct 30 19:00:52.598638 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.598101    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c" event={"ID":"73f85af5-d985-4e04-8482-f93a59d14d79","Type":"ContainerDied","Data":"5e6243d295002efe9b8e4bbc039a059b81974e05768bbf92a11f69b1aa0f2e89"}
Oct 30 19:00:52.598638 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.598239    5746 scope.go:122] "RemoveContainer" containerID="f831a4f215b1bbcc62116b2d61739cb9354d9fd6870e6b63dbcdd7e6e02f9207"
Oct 30 19:00:52.599575 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.599478    5746 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ods-monitoring/data-science-perses-0"
Oct 30 19:00:52.599575 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.599490    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="redhat-ods-monitoring/data-science-perses-0" event={"ID":"88ce202b-f466-4add-b735-dd61583d4f6f","Type":"ContainerDied","Data":"3235155d2fc723b1f856be6a1af195e6302c57d0d017ef38bbcc83a48ceaf5d7"}
Oct 30 19:00:52.606509 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.606494    5746 scope.go:122] "RemoveContainer" containerID="80a3edecd2bd2d64df1572deceb737f28abce0bcf40f9030678e2cb325506ccc"
Oct 30 19:00:52.647864 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.647837    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 19:00:52.659540 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.659512    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-perses-0"]
Oct 30 19:00:52.686287 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.686260    5746 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"]
Oct 30 19:00:52.688713 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:52.688687    5746 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["redhat-ods-monitoring/data-science-collector-targetallocator-5544bffbc5-ldh4c"]
Oct 30 19:00:54.007929 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:54.007870    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73f85af5-d985-4e04-8482-f93a59d14d79" path="/var/lib/kubelet/pods/73f85af5-d985-4e04-8482-f93a59d14d79/volumes"
Oct 30 19:00:54.008365 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:00:54.008282    5746 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88ce202b-f466-4add-b735-dd61583d4f6f" path="/var/lib/kubelet/pods/88ce202b-f466-4add-b735-dd61583d4f6f/volumes"
Oct 30 19:01:00.141384 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:00.141351    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:01:10.140580 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:10.140535    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:01:12.789664 ip-10-0-131-31 kubenswrapper[5746]: E1030 19:01:12.789626    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 19:01:20.140161 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:20.140124    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:01:23.783342 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:23.783314    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-trnpq_08444539-c0d4-4b26-9bbd-c75c21b32a21/global-pull-secret-syncer/0.log"
Oct 30 19:01:24.532569 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:24.532537    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s6687_a304069c-7b5f-4ddf-9549-33797ac13658/konnectivity-agent/0.log"
Oct 30 19:01:24.825750 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:24.825678    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-31.ec2.internal_167496dc01702046817db5cdac7ef75c/haproxy/0.log"
Oct 30 19:01:30.140204 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:30.140164    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:01:32.618417 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.618336    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7x2c_2eeef7c4-96d5-4b07-8e8d-d8dff6385427/node-exporter/0.log"
Oct 30 19:01:32.734303 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734273    5746 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"]
Oct 30 19:01:32.734555 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734542    5746 cpu_manager.go:414] "containerMap: removing container" podUID="ce3472e4-6b27-465f-bd8a-821eb643251c" containerName="thanos-querier"
Oct 30 19:01:32.734555 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734558    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce3472e4-6b27-465f-bd8a-821eb643251c" containerName="thanos-querier"
Oct 30 19:01:32.734677 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734571    5746 cpu_manager.go:414] "containerMap: removing container" podUID="73f85af5-d985-4e04-8482-f93a59d14d79" containerName="ta-container"
Oct 30 19:01:32.734677 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734578    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73f85af5-d985-4e04-8482-f93a59d14d79" containerName="ta-container"
Oct 30 19:01:32.734677 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734596    5746 cpu_manager.go:414] "containerMap: removing container" podUID="88ce202b-f466-4add-b735-dd61583d4f6f" containerName="perses"
Oct 30 19:01:32.734677 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734604    5746 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88ce202b-f466-4add-b735-dd61583d4f6f" containerName="perses"
Oct 30 19:01:32.734845 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734701    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce3472e4-6b27-465f-bd8a-821eb643251c" containerName="thanos-querier"
Oct 30 19:01:32.734845 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734712    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="73f85af5-d985-4e04-8482-f93a59d14d79" containerName="ta-container"
Oct 30 19:01:32.734845 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.734722    5746 memory_manager.go:365] "RemoveStaleState removing state" podUID="88ce202b-f466-4add-b735-dd61583d4f6f" containerName="perses"
Oct 30 19:01:32.737821 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.737802    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.747807 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.747789    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7x2c_2eeef7c4-96d5-4b07-8e8d-d8dff6385427/kube-rbac-proxy/0.log"
Oct 30 19:01:32.757791 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.757765    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"]
Oct 30 19:01:32.831317 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.831284    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ht9cb\"/\"openshift-service-ca.crt\""
Oct 30 19:01:32.831317 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.831311    5746 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ht9cb\"/\"kube-root-ca.crt\""
Oct 30 19:01:32.831521 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.831356    5746 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ht9cb\"/\"default-dockercfg-b858q\""
Oct 30 19:01:32.862313 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.862291    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-lib-modules\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.862449 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.862320    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpvb8\" (UniqueName: \"kubernetes.io/projected/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-kube-api-access-lpvb8\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.862449 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.862348    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-proc\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.862449 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.862372    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-podres\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.862616 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.862448    5746 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-sys\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.884004 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.883947    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7x2c_2eeef7c4-96d5-4b07-8e8d-d8dff6385427/init-textfile/0.log"
Oct 30 19:01:32.963763 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963733    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-lib-modules\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.963900 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963765    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpvb8\" (UniqueName: \"kubernetes.io/projected/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-kube-api-access-lpvb8\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.963900 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963792    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-proc\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.963900 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963811    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-podres\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.963900 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963842    5746 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-sys\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.963900 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963877    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-lib-modules\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.963900 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963884    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-proc\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.964163 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963954    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-podres\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.964163 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.963969    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-sys\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:32.986698 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:32.986675    5746 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpvb8\" (UniqueName: \"kubernetes.io/projected/aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3-kube-api-access-lpvb8\") pod \"perf-node-gather-daemonset-p2sjb\" (UID: \"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3\") " pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:33.049858 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.049826    5746 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:33.344746 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.344677    5746 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"]
Oct 30 19:01:33.348115 ip-10-0-131-31 kubenswrapper[5746]: W1030 19:01:33.348089    5746 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaa34b1b9_8ece_4d17_a8ae_b1eae1b84fa3.slice/crio-3eadd03b94f1f71f27b484410eade6fd9c7035316f3efb654fa2debfb6fad161 WatchSource:0}: Error finding container 3eadd03b94f1f71f27b484410eade6fd9c7035316f3efb654fa2debfb6fad161: Status 404 returned error can't find the container with id 3eadd03b94f1f71f27b484410eade6fd9c7035316f3efb654fa2debfb6fad161
Oct 30 19:01:33.419321 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.419294    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_acd7341b-f7d5-4198-a608-37bf693bbc5b/prometheus/0.log"
Oct 30 19:01:33.549434 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.549411    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_acd7341b-f7d5-4198-a608-37bf693bbc5b/config-reloader/0.log"
Oct 30 19:01:33.681006 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.680984    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_acd7341b-f7d5-4198-a608-37bf693bbc5b/thanos-sidecar/0.log"
Oct 30 19:01:33.765847 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.765815    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb" event={"ID":"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3","Type":"ContainerStarted","Data":"d20f4bee53d20cb45129311568156787f0a81e7213d45279b97ddbd994f93a8b"}
Oct 30 19:01:33.766031 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.765865    5746 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb" event={"ID":"aa34b1b9-8ece-4d17-a8ae-b1eae1b84fa3","Type":"ContainerStarted","Data":"3eadd03b94f1f71f27b484410eade6fd9c7035316f3efb654fa2debfb6fad161"}
Oct 30 19:01:33.766031 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.765965    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:33.807235 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.807181    5746 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb" podStartSLOduration=1.80716052 podStartE2EDuration="1.80716052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:33.806743378 +0000 UTC m=+3863.973561505" watchObservedRunningTime="2026-10-30 19:01:33.80716052 +0000 UTC m=+3863.973978647"
Oct 30 19:01:33.812723 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.812696    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_acd7341b-f7d5-4198-a608-37bf693bbc5b/kube-rbac-proxy-web/0.log"
Oct 30 19:01:33.955868 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:33.955789    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_acd7341b-f7d5-4198-a608-37bf693bbc5b/kube-rbac-proxy/0.log"
Oct 30 19:01:34.086900 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:34.086874    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_acd7341b-f7d5-4198-a608-37bf693bbc5b/kube-rbac-proxy-thanos/0.log"
Oct 30 19:01:34.218838 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:34.218761    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_acd7341b-f7d5-4198-a608-37bf693bbc5b/init-config-reloader/0.log"
Oct 30 19:01:34.358086 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:34.358058    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-5bn4n_9ca9cc19-6578-4e8d-a424-61ed63f9b562/prometheus-operator/0.log"
Oct 30 19:01:34.485684 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:34.485609    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-5bn4n_9ca9cc19-6578-4e8d-a424-61ed63f9b562/kube-rbac-proxy/0.log"
Oct 30 19:01:34.749080 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:34.749008    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69ffd7f44c-6nfl7_0aac7377-2d25-4744-8fbf-49b60ba4d2c0/telemeter-client/0.log"
Oct 30 19:01:34.880715 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:34.880682    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69ffd7f44c-6nfl7_0aac7377-2d25-4744-8fbf-49b60ba4d2c0/reload/0.log"
Oct 30 19:01:35.012019 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:35.011935    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69ffd7f44c-6nfl7_0aac7377-2d25-4744-8fbf-49b60ba4d2c0/kube-rbac-proxy/0.log"
Oct 30 19:01:39.779151 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:39.779125    5746 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ht9cb/perf-node-gather-daemonset-p2sjb"
Oct 30 19:01:40.140717 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:40.140677    5746 prober.go:122] "Probe failed" probeType="Readiness" pod="redhat-ods-monitoring/data-science-lokistack-ingester-0" podUID="516fe8f3-5f12-4635-8c44-ba5c1051dc3e" containerName="loki-ingester" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:01:42.491004 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:42.490977    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2ljs5_a6ee33fb-66e9-4483-b3c1-a289e9620de4/dns/0.log"
Oct 30 19:01:42.626832 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:42.626804    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2ljs5_a6ee33fb-66e9-4483-b3c1-a289e9620de4/kube-rbac-proxy/0.log"
Oct 30 19:01:42.790097 ip-10-0-131-31 kubenswrapper[5746]: E1030 19:01:42.790008    5746 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zwh9d" containerName="extractor"
Oct 30 19:01:43.512692 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:43.512660    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxg8q_04e386bc-2d2b-438e-bb10-af8b05ea5b30/dns-node-resolver/0.log"
Oct 30 19:01:45.005975 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:45.005951    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vxvtw_91a3053e-bb6f-4678-b865-4bb0e24f7dad/node-ca/0.log"
Oct 30 19:01:47.721396 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:47.721363    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lvffz_2c916f79-ff5c-4b5d-a1ba-a1df39c79a6f/serve-healthcheck-canary/0.log"
Oct 30 19:01:49.803783 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:49.803754    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zwh9d_0ffa15d5-c851-4b6e-bfc7-faa42f332f0b/kube-rbac-proxy/0.log"
Oct 30 19:01:49.935259 ip-10-0-131-31 kubenswrapper[5746]: I1030 19:01:49.935230    5746 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zwh9d_0ffa15d5-c851-4b6e-bfc7-faa42f332f0b/exporter/0.log"